Solid-state imaging device, method for processing signal of solid-state imaging device, and imaging apparatus

ABSTRACT

A solid-state imaging device includes a color filter unit disposed on a pixel array unit including pixels two-dimensionally arranged in a matrix and a conversion processing unit disposed on a substrate having the pixel array unit thereon. The color filter unit has a color arrangement in which a color serving as a primary component of a luminance signal is arranged in a checkerboard pattern and a plurality of colors serving as color information components are arranged in the other area of the checkerboard pattern. The conversion processing unit converts signals that are output from the pixels of the pixel array unit and that correspond to the color arrangement of the color filter unit into signals that correspond to a Bayer arrangement and outputs the converted signals.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/847,957, filed Mar. 20, 2013, which is a continuation of U.S. patentapplication Ser. No. 12/630,988, filed Dec. 4, 2009, now U.S. Pat. No.8,436,925, which claims priority to Japanese Patent Application Nos.2008-311694 and 2008-311695, filed in the Japan Patent Office on Dec. 8,2008, the entire disclosures of which are hereby incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a solid-state imaging device, a methodfor processing a signal of the solid-state imaging device, and animaging apparatus.

2. Description of the Related Art

In order to increase the sensitivity of solid-state imaging devices, aplurality of techniques in terms of a color filter array and signalprocessing of the color filter array have been developed (refer to, forexample, Japanese Unexamined Patent Application Publication No.2007-287891). One of the color filter arrays is a color filter arraythat uses a color (e.g., white (W)) serving as a primary component of aluminance signal. As color coding using a white color, whitecheckerboard color coding in which white is arranged in a checkerboardpattern is frequently used.

The output voltage of a color filter array using a white color is higherthan that of a color filter array having an RGB Bayer arrangement thathas been widely used. Accordingly, the sensitivity of a solid-stateimaging device can be increased. Note that, in the RGB Bayerarrangement, green (G) is arranged in a checkerboard pattern. Red (R)and blue (B) are also arranged in a checkerboard pattern in the otherarea of the checkerboard pattern.

In solid-state imaging apparatuses using a color filter of an RGB Bayerarrangement, in order to convert an RGB signal into a YUV signal (Y: aluminance signal, U and V: color difference signals), computation forgenerating a Y signal is necessary. In the computation, for example, thefollowing equation can be used:Y=0.29891×R+0.58661×G+0.11448×B

In general, this computation is performed by a digital signal processor(DSP) provided outside a substrate (a sensor chip) of the solid-stateimaging apparatus. Accordingly, even in the solid-state imaging devicesusing a color filter array including a white color, computation forgenerating a luminance signal Y is performed by a DSP provided outside asensor chip.

SUMMARY OF THE INVENTION

However, in signal processing performed by a solid-state imaging deviceincluding a color filter array including a white color, it is difficultto use existing DSPs designed for the RGB Bayer arrangement.Accordingly, it is necessary that a new DSP be developed if color codingis changed. If the DSP designed for the RGB Bayer arrangement is changedto a DSP for a white checkerboard pattern, enormous development cost isnecessary. Since this development cost is reflected in the price of acamera module including the DSP, it is difficult to reduce the cost ofthe camera module. Consequently, the widespread use of color codingincluding a white color is hampered.

Accordingly, the present invention provides a solid-state imagingdevice, a method for processing a signal of the solid-state imagingdevice, and an imaging apparatus capable of using existing RGB Bayerarrangement DSPs when color coding in which a color serving as a primarycomponent of a luminance signal is arranged in a checkerboard pattern isused.

In addition, by using filters of a white color which is a primarycomponent of a luminance signal for a color filter array, thesensitivity of a solid-state imaging device can be increased.Furthermore, by improving the color arrangement or the signal processingmethod, the sensitivity of the color filter array using a white filtercan be increased with a minimal decrease in resolution.

Accordingly, the present invention provides a solid-state imaging deviceincluding a color filter array of a novel color arrangement that canincrease the sensitivity with a minimal decrease in resolution, a methodfor processing a signal of the solid-state imaging device, and animaging apparatus including the solid-state imaging device.

According to an embodiment of the present invention, a solid-stateimaging device includes a color filter unit disposed on a pixel arrayunit including pixels two-dimensionally arranged in a matrix, where thecolor filter unit has a color arrangement in which a color serving as aprimary component of a luminance signal is arranged in a checkerboardpattern and a plurality of colors serving as color informationcomponents are arranged in the other area of the checkerboard pattern.The solid-state imaging device has a configuration in which signals thatare output from the pixels of the pixel array unit and that correspondto the color arrangement of the color filter unit are converted intosignals that correspond to a Bayer arrangement on a substrate having thepixel array unit thereon.

In the above-described configuration, since the color serving as theprimary component of a luminance signal is arranged in a checkerboardpattern, signals of other colors of pixels adjacent to the color in thevertical direction and the horizontal direction can be restored usingthe signal of the color serving as the primary component of a luminancesignal. Consequently, the efficiency of conversion from signalscorresponding to the color arrangement of the color filter unit intosignals corresponding to the Bayer arrangement can be increased. Inaddition, by outputting the signals corresponding to the Bayerarrangement from the substrate (a sensor chip) having the pixel arrayunit thereon, an existing DSP for the Bayer arrangement can be used as adownstream signal processing unit.

According to another embodiment of the present invention, a solid-stateimaging device includes a color filter unit disposed on a pixel arrayunit including pixels two-dimensionally arranged in a matrix, where thecolor filter unit has a color arrangement in which filters of a firstcolor serving as a primary component of a luminance signal are arrangedin a checkerboard pattern, and filters of a second color serving as aprimary component of the luminance signal for a series of four pixelsform a group, and the groups are arranged so as to form a stripe patternin one of a diagonal direction, a vertical direction, and a horizontaldirection. The solid-state imaging device has a configuration to receivesignals that are output from the pixels of the pixel array unit and thatcorrespond to the color arrangement of the color filter unit and add asignal of a pixel of a filter of the second color adjacent to the pixelof a filter of the first color to a signal of the pixel of the filter ofthe first color.

The filters of the first and second colors serving as primary componentsof a luminance signal have a sensitivity higher than those of the othercolors. Accordingly, in the color arrangement in which the filters ofthe first color are arranged in a checkerboard pattern, a series of fourfilters of the second color form a group, and the groups are arranged soas to form a stripe pattern in one of a diagonal direction, a verticaldirection, and a horizontal direction, by adding the signal of a pixelhaving the filter of the second color adjacent to a pixel having thefilter of the first color to the signal of the pixel having the filterof the first color and using the sum as a primary component of aluminance signal, the intensity of the luminance signal can beincreased.

According to the embodiments of the present invention, even when thecolor coding is changed, an existing DSP for RGB Bayer arrangement canbe still used. Accordingly, the development of a new DSP that issignificantly costly is not necessary.

In addition, according to the embodiments of the present invention, thesignal of a pixel having a filter of a second color adjacent to a pixelhaving a filter of a first color is added to the signal of the pixelhaving a filter of the first color, and the sum is used as a primarycomponent of a luminance signal. Thus, the intensity of the luminancesignal can be increased. As a result, the sensitivity can be increasedwith a minimal decrease in resolution.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration of an exemplary system configurationof a CMOS image sensor according to first and second exemplaryembodiments of the present invention;

FIG. 2 is a circuit diagram illustrating an exemplary circuitconfiguration of a unit pixel;

FIG. 3 is a circuit diagram illustrating an exemplary configuration of acircuit that allows pixel addition for four neighboring pixels to beperformed in the pixels;

FIG. 4 is a color arrangement diagram illustrating color codingaccording to a first example of the first exemplary embodiment;

FIG. 5 is a color arrangement diagram illustrating color codingaccording to a second example of the first exemplary embodiment;

FIG. 6 is a color arrangement diagram illustrating color codingaccording to a third example of the first exemplary embodiment;

FIG. 7 is a color arrangement diagram illustrating color codingaccording to a fourth example of the first exemplary embodiment;

FIG. 8 is a color arrangement diagram illustrating color codingaccording to a fifth example of the first exemplary embodiment;

FIG. 9 is a color arrangement diagram illustrating color codingaccording to a sixth example of the first exemplary embodiment;

FIG. 10 is a color arrangement diagram illustrating color codingaccording to a seventh example of the first exemplary embodiment;

FIG. 11 is a color arrangement diagram illustrating color codingaccording to an eighth example of the first exemplary embodiment;

FIG. 12 is a color arrangement diagram illustrating color codingaccording to a ninth example of the first exemplary embodiment;

FIG. 13 is a flowchart illustrating an exemplary process flow of a colorconversion process 1 performed in a high luminance mode at a time offull scanning in the case of the color coding according to the firstexample of the first exemplary embodiment and a first example of asecond exemplary embodiment;

FIGS. 14A to 14D are schematic illustrations of the color conversionprocess 1 performed in a high luminance mode at a time of full scanningin the case of the color coding according to the first example of thefirst exemplary embodiment and a first example of a second exemplaryembodiment;

FIG. 15 is a flowchart illustrating an exemplary process flow of a colorconversion process 2 performed in a low luminance mode at a time of fullscanning in the case of the color coding according to the first exampleof the first exemplary embodiment and a first example of a secondexemplary embodiment;

FIGS. 16A to 16D are schematic illustrations of the color conversionprocess 2 performed in the low luminance mode at a time of full scanningin the case of the color coding according to the first example of thefirst exemplary embodiment and a first example of a second exemplaryembodiment;

FIG. 17 is a flowchart illustrating an exemplary process flow of a pixeladdition process 1 in the case of the color coding according to thefirst example of the first exemplary embodiment;

FIGS. 18A to 18D are schematic illustrations of the pixel additionprocess 1 in the case of the color coding according to the first exampleof the first exemplary embodiment;

FIG. 19 illustrates FD addition and counter addition.

FIG. 20 is a flowchart illustrating an exemplary process flow of a pixeladdition process 2 performed in the case of the color coding accordingto the first example of the first exemplary embodiment and the firstexample of the second exemplary embodiment;

FIGS. 21A to 21D are schematic illustrations of the pixel additionprocess 2 performed in the case of the color coding according to thefirst example of the first exemplary embodiment and the first example ofthe second exemplary embodiment;

FIG. 22 is a flowchart illustrating an exemplary process flow of a pixeladdition process 3 performed in the case of the color coding accordingto the first example of the first exemplary embodiment;

FIGS. 23A to 23C are schematic illustrations of the pixel additionprocess 3 performed in the case of the color coding according to thefirst example of the first exemplary embodiment;

FIGS. 24A to 24D are schematic illustrations of a color conversionprocess performed at a time of full scanning in the case of the colorcoding according to the second example of the first exemplaryembodiment;

FIGS. 25A to 25D are schematic illustrations of a pixel addition processperformed in the case of the color coding according to the secondexample of the first exemplary embodiment;

FIGS. 26A to 26D are schematic illustrations of a color conversionprocess performed at a time of full scanning in the case of the colorcoding according to a third example of the first exemplary embodiment;

FIGS. 27A to 27E are schematic illustrations of a pixel addition processperformed in the case of the color coding according to the third exampleof the first exemplary embodiment;

FIGS. 28A to 28D are schematic illustrations of a color conversionprocess performed at a time of full scanning in the case of the colorcoding according to a fourth example of the first exemplary embodiment;

FIGS. 29A to 29F are schematic illustrations of a first type of pixeladdition process performed in the case of the color coding according tothe fourth example of the first exemplary embodiment;

FIGS. 30A to 30D are schematic illustrations of a second type of pixeladdition process performed in the case of the color coding according tothe fourth example of the first exemplary embodiment;

FIGS. 31A to 31C are schematic illustrations of a third type of pixeladdition process performed in the case of the color coding according tothe fourth example of the first exemplary embodiment;

FIGS. 32A to 32D are schematic illustrations of a fourth type of pixeladdition process performed in the case of the color coding according tothe fourth example of the first exemplary embodiment;

FIGS. 33A to 33D are schematic illustrations of a color conversionprocess performed at a time of full scanning in the case of the colorcoding according to a fifth example of the first exemplary embodiment;

FIGS. 34A to 34E are schematic illustrations of a pixel addition processperformed in the case of the color coding according to the fifth exampleof the first exemplary embodiment;

FIGS. 35A to 35D are schematic illustrations of a color conversionprocess 1 performed in the case of the color coding according to a sixthexample of the first exemplary embodiment;

FIGS. 36A to 36D are schematic illustrations of a color conversionprocess 2 performed in the case of the color coding according to thesixth example of the first exemplary embodiment;

FIGS. 37A to 37D are schematic illustrations of a pixel addition process1 performed in the case of the color coding according to the sixthexample of the first exemplary embodiment;

FIGS. 38A to 38D are schematic illustrations of a pixel addition process2 performed in the case of the color coding according to the sixthexample of the first exemplary embodiment;

FIGS. 39A to 39D are schematic illustrations of a color conversionprocess performed at a time of full scanning in the case of the colorcoding according to a seventh example of the first exemplary embodiment;

FIGS. 40A to 40E are schematic illustrations of a pixel addition processperformed in the case of the color coding according to the seventhexample of the first exemplary embodiment;

FIGS. 41A to 41D are schematic illustrations of a color conversionprocess performed at a time of full scanning in the case of the colorcoding according to an eighth example of the first exemplary embodiment;

FIGS. 42A to 42D are schematic illustrations of a pixel addition processperformed in the case of the color coding according to the eighthexample of the first exemplary embodiment;

FIGS. 43A to 43C are schematic illustrations of a color conversionprocess performed at a time of full scanning in the case of the colorcoding according to a ninth example of the first exemplary embodiment;

FIGS. 44A to 44B are schematic illustrations of a pixel addition processperformed in the case of the color coding according to the ninth exampleof the first exemplary embodiment;

FIG. 45 is a color arrangement diagram illustrating color codingaccording to a first example of the second exemplary embodiment;

FIG. 46 is a color arrangement diagram illustrating color codingaccording to a second example of the second exemplary embodiment;

FIG. 47 is a flowchart illustrating an exemplary process flow of a pixeladdition process 1 in the case of the color coding according to thefirst example of the second exemplary embodiment;

FIGS. 48A to 48D are schematic illustrations of the color conversionprocess 1 in the case of the color coding according to the first exampleof the second exemplary embodiment;

FIGS. 49A to 49D are schematic illustrations of a color conversionprocess 1 in the case of color coding according to a second example ofthe second exemplary embodiment;

FIGS. 50A to 50D are schematic illustrations of a color conversionprocess 2 in the case of color coding according to the second example ofthe second exemplary embodiment;

FIGS. 51A to 51D are schematic illustrations of a pixel addition process1 in the case of the color coding according to the second example of thesecond exemplary embodiment;

FIGS. 52A to 52D are schematic illustrations of a pixel addition process2 for color coding according to the second example of the secondexemplary embodiment;

FIG. 53 is a schematic illustration of an exemplary system configurationof a CMOS image sensor according to a third exemplary embodiment of thepresent invention;

FIG. 54 is a flowchart illustrating an exemplary process flow of a colorconversion process 1 performed in a high luminance mode at a time offull scanning in the case of the color coding according to the firstexample of the third exemplary embodiment;

FIGS. 55A to 55C are schematic illustrations of the color conversionprocess 1 performed at a time of full scanning in the case of the colorcoding according to the first example of the third exemplary embodiment;

FIG. 56 is a flowchart illustrating an exemplary process flow of a colorconversion process 2 performed in a low luminance mode at a time of fullscanning in the case of the color coding according to the first exampleof the third exemplary embodiment;

FIGS. 57A to 57C are schematic illustrations of the color conversionprocess 2 performed at a time of full scanning in the case of the colorcoding according to the first example of the third exemplary embodiment;

FIG. 58 is a flowchart illustrating an exemplary process flow of a pixeladdition process 1 in the case of the color coding according to thefirst example of the third exemplary embodiment;

FIGS. 59A to 59C are schematic illustrations of the pixel additionprocess 1 performed in the case of the color coding according to thefirst example of the third exemplary embodiment;

FIG. 60 is a flowchart illustrating an exemplary process flow of a pixeladdition process 2 in the case of the color coding according to thefirst example of the third exemplary embodiment;

FIGS. 61A to 61C are schematic illustrations of the pixel additionprocess 2 performed in the case of the color coding according to thefirst example of the third exemplary embodiment;

FIGS. 62A and 62B are schematic illustrations of a color conversionprocess 1 performed in the case of the color coding according to asecond example of the third exemplary embodiment;

FIGS. 63A to 63C are schematic illustrations of a color conversionprocess 2 performed in the case of the color coding according to thesecond example of the third exemplary embodiment;

FIGS. 64A and 64B are schematic illustrations of a pixel additionprocess 1 performed in the case of the color coding according to thesecond example of the third exemplary embodiment;

FIGS. 65A to 65C are schematic illustrations of a pixel addition process2 performed in the case of the color coding according to the secondexample of the third exemplary embodiment;

FIG. 66 is a color arrangement diagram illustrating color codingaccording to a modification of the first example;

FIG. 67 is a color arrangement diagram illustrating color codingaccording to a modification of the second example; and

FIG. 68 is a block diagram of an exemplary configuration of an imagingapparatus according to an embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Various exemplary embodiments of the present invention are described indetail below with reference to the accompanying drawings. Thedescriptions are made in the following order:

1. Exemplary Embodiment

1-1. System Configuration

1-2. Color Coding of Color Filter Array

1-3. Example of Color Coding

1-4. Sensitivity Ratio W:G:R:B

1-5. Color Conversion Process

2. Example of Application (Imaging Apparatus)

1. First Exemplary Embodiment

1-1. System Configuration

FIG. 1 is a schematic illustration of an exemplary system configurationof a solid-state imaging device (e.g., a CMOS image sensor, which is anexample of an X-Y addressing solid-state imaging device) according to afirst exemplary embodiment of the present invention.

According to the first embodiment, a CMOS image sensor 10 includes asemiconductor substrate (hereinafter also referred to as a “sensorchip”) 11. The sensor chip 11 includes a pixel array unit 12 formedthereon and a peripheral circuit unit integrated thereon. For example,the peripheral circuit unit includes a vertical drive unit 13, a columnprocessing unit 14, a horizontal drive unit 15, a conversion processingunit 16, and a system control unit 17.

The pixel array unit 12 includes a plurality of unit pixels (not shown),each including a photoelectric conversion element, two-dimensionallyarranged in an array. The unit pixel (hereinafter also simply referredto as a “pixel”) photoelectrically converts visible light incidentthereon into electrical charge in accordance with the intensity of thevisible light. A color filter array 30 is provided on the pixel arrayunit 12 on the side of a light receiving surface (a light incidentsurface). One of the key features of the present exemplary embodiment isthe color coding of the color filter array 30. The color coding of thecolor filter array 30 is described in more detail below.

Furthermore, in the pixel array unit 12, a pixel drive line 18 isdisposed in the left-right direction of FIG. 1 (a direction in which thepixels of a pixel row are arranged or the horizontal direction) for eachof the rows of the pixel array. Similarly, a vertical signal line 19 isdisposed in the up-down direction of FIG. 1 (a direction in which thepixels of a pixel column are arranged or the vertical direction) foreach of the columns of the pixel array. In FIG. 1, while only one pixeldrive line 18 is illustrated, the number of the pixel drive lines 18 isnot limited to one. One end of the pixel drive line 18 is connected toan output terminal corresponding to one of the rows of the verticaldrive unit 13.

For example, the vertical drive unit 13 includes a shift register and anaddress decoder. Although the detailed configuration thereof is notshown in FIG. 1, the vertical drive unit 13 includes a readout scanningsystem and a sweeping scanning system. The readout scanning systemsequentially scans the unit pixels from which signals are read by arow-by-row basis.

In contrast, prior to the readout scanning operation of the readout rowperformed by the readout scanning system by the time determined by ashutter speed, the sweeping scanning system performs sweeping scanningso that unnecessary electrical charge is swept (reset) out of thephotoelectric conversion elements of the unit pixels in the readout row.By sweeping (resetting) the unnecessary electrical charge using thesweeping scanning system, a so-called electronic shutter operation isperformed. That is, in the electronic shutter operation, thephotocharges of the photoelectric conversion element is discarded, and anew exposure operation (accumulation of light electrical charge) isstarted.

A signal read through a readout operation performed by the readoutscanning system corresponds to the amount of light made incident afterthe immediately previous readout operation or electronic shutteroperation is performed. In addition, a period of time from a readouttime point of the immediately previous readout operation or a sweepingtime point of the electronic shutter operation to the readout time pointof the current readout operation corresponds to an accumulation time (anexposure time) of the light electrical charge in the unit pixel.

A signal output from each of the unit pixels in the pixel row selectedand scanned by the vertical drive unit 13 is supplied to the columnprocessing unit 14 via the corresponding one of the vertical signallines 19. For each of the pixel columns of the pixel array unit 12, thecolumn processing unit 14 performs predetermined signal processing onthe analog pixel signal output from the pixel in the selected row.

An example of the signal processing performed by the column processingunit 14 is a correlated double sampling (CDS) process. In the CDSprocess, the reset level and the signal level output from each of thepixels in the selected row are retrieved, and the difference between thelevels is computed. Thus, the signals of the pixels in one of the rowsare obtained. In addition, fixed pattern noise of the pixels is removed.The column processing unit 14 may have an analog-to-digital (A/D)conversion function for converting the analog pixel signal into adigital format.

For example, the horizontal drive unit 15 includes a shift register andan address decoder. The horizontal drive unit 15 sequentially selectsand scans a circuit portion corresponding to a pixel column of thecolumn processing unit 14. Each of the pixel columns is sequentiallyprocessed by the column processing unit 14 through the selectionscanning operation performed by the horizontal drive unit 15 and issequentially output.

The conversion processing unit 16 performs computation and convertssignals corresponding to the color arrangement of the color filter array(the color filter unit) 30 and output from the pixels of the pixel arrayunit 12 into signals corresponding to the Bayer arrangement. Another ofthe key features of the present embodiment is that the conversionprocessing unit 16 is mounted on the substrate on which the pixel arrayunit 12 is formed, that is, the sensor chip 11, a color conversionprocess is performed in the sensor chip 11, and a signal correspondingto the Bayer arrangement is output from the sensor chip 11. The colorconversion process performed by the conversion processing unit 16 isdescribed in more detail below.

As widely used, the term “Bayer arrangement” represents a colorarrangement in which a color serving as a primary color informationcomponent of a luminance signal for high resolution is arranged in acheckerboard pattern, and the other two colors serving as colorinformation components of the luminance signal for not-so-highresolution are arranged in the other area of the checkerboard pattern.In a basic color coding form of the Bayer arrangement, green (G) thathas high contribution of the luminance signal is arranged in acheckerboard pattern, and red (R) and blue (B) are arranged in the otherarea of the checkerboard pattern.

The system control unit 17 receives a clock provided from outside thesensor chip 11 and data for indicating an operating mode. In addition,the system control unit 17 outputs data representing internalinformation of the CMOS image sensor 10. Furthermore, the system controlunit 17 includes a timing generator that generates a variety of timingsignals. The system control unit 17 controls driving of the verticaldrive unit 13, the column processing unit 14, the horizontal drive unit15, and the conversion processing unit 16 using the variety of timingsignals generated by the timing generator.

Circuit Configuration of Unit Pixel

FIG. 2 is an exemplary circuit diagram of a unit pixel 20. As shown inFIG. 2, the unit pixel 20 illustrated in the exemplary circuit diagramincludes a photoelectric conversion element (e.g., a photodiode 21) andfour transistors (e.g., a transfer transistor 22, a reset transistor 23,an amplifying transistor 24, and a selection transistor 25).

In this example, N-channel MOS transistors are used for the transfertransistor 22, the reset transistor 23, the amplifying transistor 24,and the selection transistor 25. However, the combination of aconductive type using the transfer transistor 22, the reset transistor23, the amplifying transistor 24, and the selection transistor 25 isonly an example, and the combination is not limited thereto.

For example, as the pixel drive line 18, three drive lines, that is, atransfer line 181, a reset line 182, and a selection line 183 areprovided to each of the unit pixels 20 in the same pixel row. One end ofthe transfer line 181, one end of the reset line 182, and one end of theselection line 183 are connected to an output terminal corresponding toone of the pixel rows of the vertical drive unit 13.

An anode electrode of the photodiode 21 is connected to a negative powersupply (e.g., the ground). The photodiode 21 photoelectrically convertsreceived light into photocharges (photoelectrons in this exemplaryembodiment) in accordance with the amount of received light. A cathodeelectrode of the photodiode 21 is connected to the gate electrode of theamplifying transistor 24 via the transfer transistor 22. A node 26electrically connected to the gate electrode of the amplifyingtransistor 24 is referred to as a “floating diffusion (FD) unit”.

The transfer transistor 22 is connected between the cathode electrode ofthe photodiode 21 and the FD unit 26. When a transfer pulse φTRF havingan active high level (e.g., a Vdd level) (hereinafter referred to as a“high active transfer pulse”) is supplied to a gate electrode of thetransfer transistor 22 via the transfer line 181, the transfertransistor 22 is turned on. Thus, the transfer transistor 22 transfersthe photocharges photoelectrically converted by the photodiode 21 to theFD unit 26.

A drain electrode of the reset transistor 23 is connected to a pixelpower supply Vdd. The source electrode of the reset transistor 23 isconnected to the FD unit 26. Before the signal charge is transferredfrom the photodiode 21 to the FD unit 26, a high active reset pulse φRSTis supplied to a gate electrode of the reset transistor 23 via the resetline 182. When the reset pulse φRST is supplied to the reset transistor23, the reset transistor 23 is turned on. Thus, the reset transistor 23resets the FD unit 26 by discarding the electrical charge of the FD unit26 to the pixel power supply Vdd.

The gate electrode of the amplifying transistor 24 is connected to theFD unit 26. A drain electrode of the amplifying transistor 24 isconnected to the pixel power supply Vdd. After the FD unit 26 is resetby the reset transistor 23, the amplifying transistor 24 outputs thepotential of the FD unit 26 in the form of a reset signal (a resetlevel) Vreset. In addition, after the signal charge is transferred bythe transfer transistor 22, the amplifying transistor 24 outputs thepotential of the FD unit 26 in the form of a light accumulation signal(a signal level) Vsig.

For example, a drain electrode of the selection transistor 25 isconnected to the source electrode of the amplifying transistor 24. Thesource electrode of the selection transistor 25 is connected to thevertical signal line 17. When a high active selection pulse φSEL issupplied to the gate electrode of the selection transistor 25 via theselection line 163, the selection transistor 25 is turned on. Thus, theselection transistor 25 causes the unit pixel 20 to enter a selectedmode so that a signal output from the amplifying transistor 24 isrelayed to the vertical signal line 17.

Note that a circuit configuration in which the selection transistor 25is connected between the pixel power supply Vdd and the drain of theamplifying transistor 24 may be employed.

It should be noted that the pixel structure of the unit pixel 20 is notlimited to the above-described four-transistor pixel structure. Forexample, the unit pixel 20 may have a three-transistor pixel structurein which the functions of the amplifying transistor 24 and the selectiontransistor 25 are performed by one transistor. Thus, any configurationof a pixel circuit can be employed.

In general, in order to increase a frame rate when a moving image iscaptured, pixel addition in which signals output from a plurality ofneighboring pixels are summed and read out is performed. The pixeladdition is performed in a pixel, signal lines, the column processingunit 14, or a downstream signal processing unit. In the presentembodiment, for example, a pixel structure in which four pixels arrangedso as to be adjacent to one another in the vertical direction and thehorizontal direction is described.

FIG. 3 is a circuit diagram of an exemplary configuration of a circuitthat allows pixel addition for four neighboring pixels to be performedin the pixels. The same numbering will be used in describing FIG. 3 aswas utilized above in describing FIG. 2, where appropriate.

In FIG. 3, the photodiodes 21 of the four pixels arranged so as to beadjacent to one another in the vertical direction and the horizontaldirection are denoted as photodiodes 21-1, 21-2, 21-3, and 21-4. Fourtransfer transistors 22-1, 22-2, 22-3, and 22-4 are provided to thephotodiodes 21-1, 21-2, 21-3, and 21-4, respectively. In addition, theone reset transistor 23, the one amplifier transistor 24, and the oneselection transistor 25 are used.

That is, one of the electrodes of the transfer transistor 22-1, one ofthe electrodes of the transfer transistor 22-2, one of the electrodes ofthe transfer transistor 22-3, and one of the electrodes of the transfertransistor 22-4 are connected to the cathode electrode of the photodiode21-1, the cathode electrode of the photodiode 21-2, the cathodeelectrode of the photodiode 21-2, and the cathode electrode of thephotodiode 21-2, respectively. The other electrode of the transfertransistor 22-1, the other electrode of the transfer transistor 22-2,the other electrode of the transfer transistor 22-3, and the otherelectrode of the transfer transistor 22-4 are commonly connected to thegate electrode of the amplifier transistor 24. In addition, the FD unit26 that is shared by the photodiodes 21-1, 21-2, 21-3, and 21-4 iselectrically connected to the gate electrode of the amplifier transistor24. The drain electrode of the reset transistor 23 is connected to thepixel power supply Vdd, and the source electrode of the reset transistor23 is connected to the FD unit 26.

In the above-described pixel structure that supports the pixel additionfor four neighboring pixels, by providing the transfer pulse φTRF to thefour transfer transistors 22-1, 22-2, 22-3, and 22-4 at the same time,pixel addition for four neighboring pixels can be performed. That is,the signal charges transferred from the photodiodes 21-1, 21-2, 21-3,and 21-4 to the FD unit 26 by the four transfer transistors 22-1, 22-2,22-3, and 22-4 are summed by the FD unit 26.

In contrast, by providing the transfer pulse φTRF to the four transfertransistors 22-1, 22-2, 22-3, and 22-4 at different points of time,signal output can be performed on a pixel-by-pixel basis. That is, whena moving image is captured, the frame rate can be increased byperforming pixel addition. In contrast, when a still image is captured,the resolution can be increased by independently reading the signals ofall of the pixels.

1-2. Color Coding of Color Filter Array

The color coding of the color filter array 30 which is one of thefeatures of the present exemplary embodiment is described next.

According to the present exemplary embodiment, the color filter array 30employs color coding in which a color serving as a primary colorinformation component of a luminance signal is arranged in acheckerboard pattern, and a plurality of the other colors are arrangedin the other area of the checkerboard pattern. In the presentembodiment, the primary color of a luminance signal is, for example, oneof white (W), green (G), and another spectral component of the luminancesignal.

Since a W filter has a sensitivity about twice that of a G filter (theoutput level of a W filter is higher than that of a G filter), a highS/N ratio can be obtained. However, since a W filter contains variouscolor information, a false color which is different from the originalcolor of a subject tends to appear. In contrast, although a G filter hassensitivity lower than that of a W filter, a G filter produces few falsecolors. That is, there is a trade-off between the sensitivity andgeneration of a false color.

When W filters serving as filters of a primary color informationcomponent are arranged in a checkerboard pattern, R, G, and B filtersare arranged in the other areas of the checkerboard pattern as filtersof the other color information components. In contrast, when G filtersserving as filters of a primary color information component are arrangedin a checkerboard pattern, R and B filters are arranged in the otherareas of the checkerboard pattern as filters of the other colorinformation components.

In this way, by using, for the color filter array 30, color coding inwhich W filters for the primary color of a luminance signal are arrangedin a checkerboard pattern, the sensitivity of the CMOS image sensor 10can be increased, since the W filter has a sensitivity higher than thatof a filter of another color. In contrast, by using, for the colorfilter array 30, color coding in which G filters for the primary colorof a luminance signal are arranged in a checkerboard pattern, the colorreproducibility of the CMOS image sensor 10 can be increased, since theG filter produces few false colors.

In addition, when the color filter array 30 using either one of thecolor coding methods is used, a signal corresponding to the colorarrangement is converted into a signal corresponding to the Bayerarrangement by the sensor chip 11. At that time, since the color servingas the primary component of a luminance signal is arranged in acheckerboard pattern, signals of other colors of pixels adjacent to thecolor in the vertical direction and the horizontal direction can berestored using the signal of the color serving as the primary componentof a luminance signal. Consequently, the efficiency of color conversionperformed by the conversion processing unit 16 can be increased.

Furthermore, by outputting the signals corresponding to the Bayerarrangement from the sensor chip 11, an existing DSP for the Bayerarrangement can be used as a downstream signal processing unit.Basically, the DSP for the Bayer arrangement generates a luminancesignal Y and two color difference signals U(B−Y) and V(R−Y) using thesignal output from the sensor chip 11 and corresponding to the Bayerarrangement.

In this way, since an existing DSP for the Bayer arrangement can beused, development of a new DSP that is significantly costly is notnecessary even when the color coding of the color filter array 30 ischanged. Accordingly, a camera module including a DSP can be produced ata low cost. As a result, the widespread use of the color filter array 30using, in particular, a W filter can be expected.

1-3. Examples of Color Coding of Color Filter Array

Examples of color coding that facilitate conversion from a signalcorresponding to a color arrangement in which filters of a color servingas a primary component of a luminance signal are arranged in acheckerboard pattern to a signal corresponding to an RGB Bayerarrangement are described in detail below.

First Example of First Exemplary Embodiment

FIG. 4 is a color arrangement diagram illustrating color codingaccording to a first example of the first exemplary embodiment. As shownin FIG. 4, in the color coding according to the first example of thefirst exemplary embodiment, W filters that maximize the output level arearranged in a checkerboard pattern. R filters are arranged in acheckerboard pattern at a two-pixel pitch in the vertical direction andthe horizontal direction. Similarly, B filters are arranged in acheckerboard pattern at a two-pixel pitch in the vertical direction andthe horizontal direction. Each of the R filters is diagonally shiftedfrom one of the B filters by one pixel. In addition, G filters arearranged in the other area of the checkerboard pattern.

More specifically, in a 4×4 pixel block, W filters are arranged in acheckerboard pattern. R filters are arranged in the second row and firstcolumn and in the fourth row and third column. B filters are arranged inthe first row and second column and in the third row and fourth column.This array is the checkerboard pattern having a two-pixel pitch. Inaddition, G filters are arranged in the other area of the checkerboardpattern. At that time, the G filters form a diagonal stripe pattern.

Second Example of First Exemplary Embodiment

FIG. 5 is a color arrangement diagram illustrating color codingaccording to a second example of the first exemplary embodiment. Asshown in FIG. 5, in the color coding according to the second example ofthe first exemplary embodiment, W filters are arranged in a checkerboardpattern. R filters are squarely arranged in the pattern at a four-pixelpitch in the vertical direction and the horizontal direction. Similarly,B filters are squarely arranged in the pattern at a four-pixel pitch inthe vertical direction and the horizontal direction. In addition, eachof the R filters is diagonally shifted from one of the B filters by onepixel. In addition, G filters are arranged in the other area in thecheckerboard pattern.

More specifically, in a 4×4 pixel block, W filters are arranged in acheckerboard pattern. An R filter is disposed in the second row andthird column. A B filter is disposed in the third row and second column.This array is the square array having a four-pixel pitch in the verticaldirection and the horizontal direction. In addition, G filters arearranged in the other area of the checkerboard pattern. At that time,the G filters form a diagonal stripe pattern.

Third Example of First Exemplary Embodiment

FIG. 6 is a color arrangement diagram illustrating color codingaccording to a third example of the first exemplary embodiment. As shownin FIG. 6, in the color coding according to the third example of thefirst exemplary embodiment, W filters are arranged in a checkerboardpattern. R filters are squarely arranged in a checkerboard pattern at afour-pixel pitch in the vertical direction and the horizontal direction.Similarly, B filters are squarely arranged in a checkerboard pattern ata four-pixel pitch in the vertical direction and the horizontaldirection. In addition, each of the R filters is diagonally shifted fromone of the B filters by two pixels. In addition, G filters are arrangedin the other area in the checkerboard pattern.

More specifically, in a 4×4 pixel block, W filters are arranged in acheckerboard pattern. An R filter is disposed in the second row andfirst column. A B filter is disposed in the fourth row and third column.This array is the square array having a four-pixel pitch in the verticaldirection and the horizontal direction. In addition, G filters arearranged in the other area of the checkerboard pattern. At that time,the G filters form a diagonal stripe pattern.

Fourth Example of First Exemplary Embodiment

FIG. 7 is a color arrangement diagram illustrating color codingaccording to a fourth example of the first exemplary embodiment. Asshown in FIG. 7, in the color coding according to the fourth example ofthe first exemplary embodiment, W filters are arranged in a checkerboardpattern. Filters of each of R and B are arranged in a checkerboardpattern at a two-pixel pitch in the vertical direction and thehorizontal direction. In addition, each of the R filters is diagonallyshifted from one of the B filters by one pixel. In addition, G filtersare arranged in the other area in the checkerboard pattern.

More specifically, in a 4×4 pixel block, W filters are arranged in acheckerboard pattern. R filters are disposed in the first row and secondcolumn and in the third row and fourth column. B filters are disposed inthe third row and second column and in the first row and fourth column.This array is the checkerboard array having a two-pixel pitch in thevertical direction and the horizontal direction. In addition, G filtersare arranged in the other area of the checkerboard pattern.

Fifth Example of First Exemplary Embodiment

FIG. 8 is a color arrangement diagram illustrating color codingaccording to a fifth example of the first exemplary embodiment. As shownin FIG. 8, in the color coding according to the fifth example of thefirst exemplary embodiment, W filters are arranged in a checkerboardpattern. R filters are squarely arranged in a checkerboard pattern at atwo-pixel pitch in the vertical direction and the horizontal direction.Similarly, B filters are squarely arranged in a checkerboard pattern ata two-pixel pitch in the vertical direction and the horizontaldirection. In addition, each of the R filters is diagonally shifted fromone of the B filters by one pixel.

More specifically, in a 4×4 pixel block, W filters are arranged in acheckerboard pattern. R filters are disposed in the second row and firstcolumn, the second row and third column, the fourth row and firstcolumn, and the fourth row and third column. B filters are disposed inthe first row and second column, the first row and fourth column, thethird row and second column, and the third row and fourth column. Thisarray is the square array having a two-pixel pitch in the verticaldirection and the horizontal direction.

Sixth Example of First Exemplary Embodiment

FIG. 9 is a color arrangement diagram illustrating color codingaccording to a sixth example of the first exemplary embodiment. As shownin FIG. 9, in the color coding according to the sixth example of thefirst exemplary embodiment, W filters are arranged in a checkerboardpattern. R filters are squarely arranged in a checkerboard pattern at afour-pixel pitch in the vertical direction and the horizontal direction.Similarly, B filters are squarely arranged in a checkerboard pattern ata four-pixel pitch in the vertical direction and the horizontaldirection. In addition, each of the R filters is diagonally shifted fromone of the B filters by two pixels. Furthermore, G filters are arrangedin the other area of the checkerboard pattern.

More specifically, in a 4×4 pixel block, W filters are arranged in acheckerboard pattern. R filters are disposed in the third row and fourthcolumn and in the fourth row and third column. B filters are disposed inthe first row and second column and in the second row and first column.This array is the square array having a four-pixel pitch in the verticaldirection and the horizontal direction. In addition, G filters arearranged in the other area of the checkerboard pattern. At that time,the G filters form a diagonal stripe pattern.

The color coding methods according to the above-described first to sixthexamples of the first exemplary embodiment use a color arrangement inwhich W filters having a color serving as a primary component of aluminance signal that maximizes the output level are arranged in acheckerboard pattern. Since the filters of a white color (W) thatincludes R, G, and B color components are arranged in a checkerboardpattern, the accuracy of conversion into signals corresponding to theRGB Bayer arrangement can be increased.

The key feature of these color coding methods is that if the W filtersare replaced with G filters during the color conversion processdescribed below, the locations of the R and B filters are coincidentwith those of the Bayer arrangement. In addition, for the locations atwhich colors are not coincident, information regarding pixels of the Wfilters can be used. Thus, R and B pixel information can be restored. Asa result, conversion efficiency can be significantly increased.

Another of the key features of the color coding methods according to thefirst to third examples and the sixth example of the first exemplaryembodiment is that the W filters are arranged in a checkerboard pattern,and a series of four G filters arranged in a diagonal directionrepeatedly appears so that a diagonal stripe pattern is formed. In suchcolor coding methods, by summing the signals of pixels of G filtersadjacent to a pixel of a W filter and the signal of the W filter andusing the sum as a primary component of a luminance signal, theintensity of the luminance signal can be increased. Accordingly, thesensitivity (the S/N ratio) can be increased.

In particular, in the color coding method according to the first exampleof the first embodiment, each of the R filters is diagonally shiftedfrom one of the B filters by one pixel. Accordingly, the efficiency ofconversion into a signal corresponding to the Bayer arrangement can beincreased. In addition, in the color coding method according to thesecond example of the first embodiment, R filters are arranged so as toform a square array having a four-pixel pitch in the vertical directionand the horizontal direction, and B filters are arranged so as to form asquare array having a four-pixel pitch in the vertical direction and thehorizontal direction. Furthermore, the each of the R filters isdiagonally shifted from one of the B filters by two pixels. Accordingly,the efficiency of conversion can be increased. Still furthermore, in thecolor coding method according to the second example of the firstembodiment, the number of G filters can be large. Thus, the efficiencyof conversion into the G color can be increased.

In the color coding methods according to the first and sixth examples ofthe first embodiment, a series of four G filters arranged in a diagonaldirection repeatedly appears so that a diagonal stripe pattern isformed. Accordingly, in such color coding methods, by adding the signalof a G pixel or the signals of two G pixels adjacent to a W pixel to thesignal of the W pixel and using the sum signal as a primary component ofa luminance signal, a high sensitivity (a high S/N rate) can be providedwithout a decrease in resolution. This advantage can be provided notonly when a series of four G filters arranged in a diagonal directionrepeatedly appears so that a diagonal stripe pattern is formed, but alsowhen a series of four G filters arranged in the vertical direction orthe horizontal direction repeatedly appears so that a stripe pattern isformed.

Seventh Example of First Exemplary Embodiment

FIG. 10 is a color arrangement diagram illustrating color codingaccording to a seventh example of the first exemplary embodiment. Asshown in FIG. 10, in the color coding according to the seventh exampleof the first exemplary embodiment, G filters serving as color filters ofa primary component of a luminance signal are arranged in a checkerboardpattern. R filters and B filters are arranged in a checkerboard patternat a two-pixel pitch in the vertical direction and the horizontaldirection. Each of the R filters is diagonally shifted from one of the Bfilters by two pixels. In addition, W filters are arranged in the otherarea of the checkerboard pattern.

More specifically, in an array including four pixels in each of thevertical direction and the horizontal direction, G filters are arrangedin a checkerboard pattern. R filters are arranged in the first row andfirst column and in the third row and third column. B filters arearranged in the first row and third column and in the third row andfirst column. This array is the checkerboard pattern having a two-pixelpitch in each of the vertical direction and the horizontal direction. Inaddition, W filters are arranged in the other area of the checkerboardpattern.

In the color coding methods according to the first to seventh examplesof the first embodiment, W or G filters serving as color filters of aprimary component of a luminance signal are arranged in a checkerboardpattern. However, in order to easily convert a signal corresponding tothe RGB Bayer arrangement using the conversion processing unit 16 on thesensor chip 11, color coding is not limited to the color coding in whichW or G filters are arranged in a checkerboard pattern. Color coding inwhich W or G filters are not arranged in a checkerboard pattern isdescribed below with reference to eighth and ninth examples of the firstexemplary embodiment.

Eighth Example of First Exemplary Embodiment

FIG. 11 is a color arrangement diagram illustrating color codingaccording to an eighth example of the first exemplary embodiment. Asshown in FIG. 11, in the color coding according to the eighth example ofthe first exemplary embodiment, each of 2×2 pixel blocks includes W, R,G, and B filters, and a pattern having a two-pixel pitch in the verticaldirection and the horizontal direction is formed. More specifically,each of the W filters is disposed in an even row and an even column,each of the R filters is disposed in an even row and an odd column, eachof the G filters is disposed in an odd row and an odd column, and eachof the B filters is disposed in an odd row and an even column.

Ninth Example of First Exemplary Embodiment

FIG. 12 is a color arrangement diagram illustrating color codingaccording to a ninth example of the first exemplary embodiment. As shownin FIG. 12, in the color coding according to the ninth example of thefirst exemplary embodiment, each of 2×2 pixel blocks includes R, G, or Bfilters, and these pixel blocks form the Bayer arrangement. Morespecifically, G filters are disposed in first and second rows and firstand second columns and in third and fourth rows and third and fourthrows, B filters are disposed in the first and second rows and the thirdand fourth columns, and R filters are disposed in third and fourth rowsand the first and second columns.

1-4. Sensitivity Ratio W:G:R:B

A sensitivity ratio W:G:R:B is described next. In color coding includingW filters, a W filter pixel that has a high output signal level issaturated earlier than a pixel of other color filters. Accordingly, itis necessary that a sensitivity balance among W, G, R, and B pixels besustained, that is, the sensitivity ratio W:G:R:B be adjusted bydecreasing the sensitivity of the W filter pixel and increasing thesensitivities of the other color filter pixel relative to thesensitivity of the W filter pixel.

In order to adjusting the sensitivity ratio, widely used exposurecontrol techniques can be used. More specifically, by adjusting the sizeof an on-chip microlens provided outside the color filter array 30 foreach of the pixels of the color filter array 30, the balance among theamounts of light made incident on individual color pixels is sustainable(refer to, for example, Japanese Unexamined Patent ApplicationPublication No. 9-116127). By using this technique and decreasing thesize of an on-chip microlens for a W color to a size smaller than thatfor each of the other colors, the sensitivity of the W filter pixel canbe decreased. In this way, the sensitivity of one of the other colorfilter pixel can be relatively increased.

Alternatively, an exposure light control technique in which, in colorcoding including a W filter, the difference between the sensitivitiescan be reduced by removing an on-chip microlens for a W pixel, and acolor S/N can be improved by increasing the color sensitivity can beused (refer to, for example, Japanese Unexamined Patent ApplicationPublication No. 2007-287891). By using this technique, the sensitivityof the W filter pixel can be decreased. In this way, the sensitivity ofone of the other color filter pixel can be relatively increased.

Still alternatively, in order to preventing the improper color balance,a technique of performing shutter exposure control in which the exposuretime of a G filter pixel is decreased compared with that for an R or Bfilter pixel can be used (refer to, for example, Japanese UnexaminedPatent Application Publication No. 2003-60992). By combining such ashutter exposure control technique with control of light-receiving area,the sensitivity of the W filter pixel can be decreased. In this way, thesensitivity of one of the other color filter pixel can be relativelyincreased. Furthermore, in particular, the occurrence of a coloredoutline of a moving subject can be eliminated. As a result, anachromatizing process performed by an external signal processing unit (aDSP) is not necessary.

Note that the above-described three exposure control techniques used foradjusting the sensitivity ratio are only examples. The exposure controltechnique is not limited thereto.

For example, the size of the on-chip microlens for a W pixel is set insuch a manner that the output levels of the W, G, B, and R pixels aresubstantially in the proportion 2:1:0.5:0.5.

For 1.1-μm pixels, when the size of an on-chip microlens for a W pixelis varied by ±0.1 μm, the area is doubled and halved. Therefore, thesensitivity is doubled or halved. Accordingly, the output level of a Wpixel having an area the same as that of a G pixel and having an outputlevel double that of the G pixel can be adjusted so that the outputlevels are the same. Even when the size of the on-chip microlens isvaried by ±0.05 μm, the area is ±1.42 times the original area and,therefore, the sensitivity of the W pixel can be reduced to 1.42 timesthat of the G pixel. In such a case, the further adjustment of thesensitivity may be performed by shutter exposure control.

1-5. Color Conversion Process

A process for converting a signal into a signal corresponding to the RGBBayer arrangement (i.e., a color conversion process) performed by theconversion processing unit 16 is described in more detail next.

The following two types of color conversion process are provided: acolor conversion process performed when a still image is captured (at atime of full scanning in which all of the pixels are scanned) and acolor conversion process performed when a moving image is captured (at atime of pixel addition in which signals of a plurality of pixelsneighboring a given pixel is added to the signal of the given pixel). Inthe case of color coding according to the first and sixth examples ofthe first exemplary embodiment, a color conversion process with a highsensitivity can be performed. Accordingly, a low luminance mode can beused and, therefore, the color conversion process performed at a time offull scanning can be divided into two color conversion processes.

One of the two color conversion processes is performed when theluminance of incident light is higher than a predetermined referenceluminance. This color conversion process is referred to as a “colorconversion process 1”. The other color conversion process is performedwhen the luminance of incident light is lower than or equal to thepredetermined reference luminance. This color conversion process isreferred to as a “color conversion process 2”. In addition, the colorconversion process performed at a time of pixel addition can be dividedinto a plurality of color conversion processes in accordance with thecombinations of the pixels to be added.

Note that, in the case of the color coding at a time of full scanningaccording to the examples other than the first and sixth examples, it isdifficult to use a low luminance mode. Accordingly, only a highluminance mode is used. That is, in the case of the color coding at atime of full scanning according to the second to fifth examples and theseventh to ninth examples, the color conversion process 1 used in thefirst and sixth examples for high luminance is used.

Color Coding According to First Example of First Exemplary Embodiment

A color conversion process performed for the color coding according tothe first example of the first exemplary embodiment is described next.First, the color conversion process 1 performed in a high luminance modeat a time of full scanning is described with reference to a flowchartshown in FIG. 13 and schematic illustrations shown in FIGS. 14A to 14D.

As illustrated by the flowchart shown in FIG. 13, the color conversionprocess 1 in a high luminance mode is realized by sequentiallyperforming the processing in steps S11, S12, and S13. FIG. 14Aillustrates a 4×4 pixel color arrangement of the color coding accordingto the first example.

In step S11, as shown in FIG. 14B, the components of white (W) pixelsarranged in a checkerboard pattern are expanded into pixels of allcolors by determining the direction of resolution. As used herein, theterm “direction of resolution” refers to a direction in which pixelsignals are present. In FIG. 14B, “W” surrounded by a square framerepresents a component of a W pixel after the component of the W pixelis expanded into each of all colors.

In order to expand a component of a W pixel into pixels of other colors,signal processing based on a widely used directional correlation can beapplied. For example, in signal processing based on a directionalcorrelation, a plurality of color signals corresponding to a given pixelis acquired, and the correlation value in the vertical direction and/orthe horizontal direction is obtained (refer to, for example, JapanesePatent No. 2931520).

In step S12, as shown in FIG. 14C, a W pixel is replaced with a G pixelusing a correlation between a W pixel and a G pixel. As can be seen fromthe above-described color arrangements of various color coding, a Wpixel is adjacent to a G pixel. In terms of a correlation between a Wpixel and a G pixel in a certain area, the W pixel and the G pixel havea strong correlation since either one of the W pixel and the G pixel hasa color serving as a primary component of a luminance signal. Thus, thecorrelation value (the correlation coefficient) is nearly 1. Bydetermining the direction of resolution using the color correlation andchanging the output level of a W pixel to the level equivalent to theoutput level of a G pixel, the W pixel is replaced with the G pixel.

In step S13, an R pixel and a B pixel are generated for the Bayerarrangement shown in FIG. 14D using a correlation between the W pixeland the R pixel and a correlation between the W pixel and the B pixel.Since a W pixel includes R, G, and B color components, the correlationbetween the W pixel and the R pixel and the correlation between the Wpixel and the B pixel can be obtained. For the signal processing, anexisting technique in which a luminance signal to be replaced with G ina four-color arrangement is generated for every pixel by interpolationcan be used (refer to, for example, Japanese Unexamined PatentApplication Publication No. 2005-160044).

Subsequently, the color conversion process 2 performed in a lowluminance mode at a time of full scanning is described with reference toa flowchart shown in FIG. 15 and schematic illustrations shown in FIGS.16A to 16D.

First, as shown in FIG. 16A, the direction of resolution is examined byusing signal processing based on the above-described widely usedtechnique (step S21). Thereafter, it is determined whether the directionof resolution can be determined (step S22). If the direction ofresolution can be determined, the components of the W pixels arranged ina checkerboard pattern are expanded into pixels of all colors (stepS23).

Subsequently, as shown in FIG. 16B, R pixels and B pixels are generatedusing the above-described widely used technique, the correlation betweena W pixel and an R pixel, and the correlation between a W pixel and a Bpixel (step S24). Thereafter, as shown in FIG. 16C, the signals of two Rpixels adjacent to the W pixel are added to the signal of the W pixel sothat the signal of the W pixel is approximated by G (=W+2G). In thisway, as shown in FIG. 16D, each of the G pixels is generated for theBayer arrangement (step S25).

However, if, in step S22, the direction of resolution is not determined,each of the R and B pixels is generated for the Bayer arrangement byusing simple four-pixel averaging interpolation using four pixelsadjacent to the W pixel in the vertical direction and the horizontaldirection (step S26).

As described above, by using the color conversion process 1 and colorconversion process 2 in accordance with the luminance of the incidentlight, signals corresponding to a color arrangement in which W filtersare arranged in a checkerboard pattern can be converted into signalscorresponding to the RGB Bayer arrangement on the sensor chip 11, andthe signals can be output.

Two color conversion processes performed at a time of pixel addition formoving image capturing are described next. Hereinafter, one of the twocolor conversion processes is referred to as a “pixel addition process1”, and the other color conversion process is referred to as a “pixeladdition process 2”.

First, the pixel addition process 1 is described with reference to aflowchart shown in FIG. 17 and schematic illustrations shown in FIGS.18A to 18D.

Addition is performed on two W pixels diagonally adjacent to each other(step S31). More specifically, as shown in FIG. 18A, addition isperformed on a pixel of interest and a pixel located to the lower rightof the pixel of interest (i.e., a pixel to the right by one column andbelow by one row). This addition can be performed in the pixel structureshown in FIG. 3 by providing the transfer pulse φTRF to the transfertransistors 22 of the two pixels which are the targets of addition (thetransfer transistors 22-1 and 22-4 in this example) at the same time. Inthis way, two-pixel addition can be performed in the FD unit 26.Hereinafter, such pixel addition is referred to as “FD addition”.

Subsequently, for each of R, G, and B pixels, addition is performed ontwo pixels arranged with one pixel therebetween in the opposite diagonaldirection (step S32). More specifically, as shown in FIG. 18B, additionis performed on a pixel of interest and a pixel located to the lowerleft of the pixel of interest with one pixel therebetween (i.e., a pixelto the right by two columns and below by two rows). For each of R, G,and B pixels, when the column processing unit 14 shown in FIG. 1 has anA/D conversion function, this addition can be performed during A/Dconversion.

More specifically, in the color arrangement shown in FIG. 19, thesignals of pixels B1 and G1 are read independently. After the signalsare A/D-converted, the signals of pixels B2 and G3 are continuously readand A/D-converted. In this way, two-pixel addition can be performed. Inorder to perform pixel addition during A/D conversion performed by thecolumn processing unit 14, an existing technique for converting ananalog pixel signal into a digital pixel signal using a counter can beused (refer to, for example, Japanese Unexamined Patent ApplicationPublication No. 2006-033454).

Hereinafter, this pixel addition performed using a counter of an A/Dconverter is referred to as “counter addition”. When counter addition isperformed and if the gain is changed on a line-by-line basis, theaddition ratio can be varied. Similarly, counter addition can beperformed on R pixels. Note that, in the above-described two-pixeladdition for the W pixels, FD addition is performed between the pixelsW1 and W2 and between the pixels W3 and W4.

Thereafter, as shown in FIG. 18C, the components of the W pixel arefitted into the R, G, and B pixels (step S33). Subsequently, as shown inFIG. 18D, four pixels for the RGB Bayer arrangement are generated (stepS34).

Subsequently, the pixel addition process 2 is described with referenceto a flowchart shown in FIG. 20 and schematic illustrations shown inFIGS. 21A to 21D.

First, for W pixels and G pixels, FD addition is performed between twopixels arranged in the upper left-bottom right diagonal directions andthe upper right-bottom left diagonal directions, respectively. Thus, W,R, G, and B pixels are generated (step S41). More specifically, for theW pixels, as shown in FIG. 21A, FD addition is performed between a pixelof interest and a pixel located at the lower right of the pixel ofinterest (i.e., a pixel to the right by one column and below by onerow). For the G pixels, as shown in FIG. 21B, FD addition is performedbetween a pixel of interest and a pixel located at the lower left of thepixel of interest (i.e., a pixel to the left by one column and below byone row).

Note that, among eight pixels of a 4×4 pixel block, a pair of R and Bsignals is not used. That is, R pixels and B pixels are read in athinning-out manner without performing pixel addition. Accordingly, thesensitivity of R and B is decreased, as compared with that in the pixeladdition process 1. Consequently, a color S/N ratio is low.

Thereafter, as shown in FIG. 21C, the components of the W pixel arefitted into the R, G, and B pixels (step S42). Subsequently, as shown inFIG. 21D, four pixels for the RGB Bayer arrangement are generated (stepS43). In the case of the pixel addition process 2, the centroid locationof the Bayer arrangement is slightly shifted from that in the pixeladdition process 1.

Subsequently, the pixel addition process 3 is described with referenceto a flowchart shown in FIG. 22 and schematic illustrations shown inFIGS. 23A to 23C.

First, for each of W, R, G, and B pixels, addition is performed betweentwo pixels arranged in a diagonal direction with a pixel therebetween(step S51). Thus, as shown in FIG. 23B, a color arrangement including arow of R, W, G, W, . . . and the next row of W, G, W, R, . . . isobtained after the addition. Thereafter, as shown in FIG. 23C, in amanner the same as that in the process at a time of full scanning, R andB pixels are generated for the Bayer arrangement using the correlationbetween a W pixel and an R pixel and the correlation between a W pixeland an R pixel (step S52).

As described above, by using one of the pixel addition process 1, thepixel addition process 2, and the pixel addition process 3 when a movingimage is captured, signals corresponding to a color arrangement in whichW filters are arranged in a checkerboard pattern can be converted intosignals corresponding to the RGB Bayer arrangement on the sensor chip11, and the signals can be output.

Color conversion processes for color coding according to the second toninth examples are described below. In many cases, the process sequenceis similar to that of the first example.

Color Coding According to Second Example of First Exemplary Embodiment

A color conversion process at a time of full scanning is described withreference to schematic illustrations shown in FIGS. 24A to 24D. In acolor arrangement of a 4×4 pixel block according to the second exampleof the first exemplary embodiment shown in FIG. 24A, the components of Wpixels arranged in a checkerboard pattern are expanded into pixels ofall colors using the direction of resolution, as shown in FIG. 24B.Subsequently, as shown in FIG. 24C, the W pixels are replaced with Gpixels using a correlation between a W pixel and a G pixel. Thereafter,as shown in FIG. 24D, R and B pixels are generated for the RGB Bayerarrangement using a correlation between a W pixel and an R pixel and acorrelation between a W pixel and a B pixel.

The pixel addition process is described next with reference to schematicillustrations shown in FIGS. 25A to 25D. As shown in FIGS. 25A and 25B,for a W pixel and a G pixel, FD addition is performed between two pixelslocated in the bottom right-upper left diagonal directions and bottomleft-upper right diagonal directions, respectively. Thus, W, R, G, and Bpixels are generated. Thereafter, as shown in FIG. 25C, the componentsof the W pixels are fitted into the R, G, and B pixels. Thus, as shownin FIG. 25D, four pixels for the RGB Bayer arrangement are generated.

Color Coding According to Third Example of First Exemplary Embodiment

A color conversion process at a time of full scanning is described withreference to schematic illustrations shown in FIGS. 26A to 26D. In acolor arrangement of a 4×4 pixel block according to the third example ofthe first exemplary embodiment shown in FIG. 26A, the components of Wpixels arranged in a checkerboard pattern are expanded into pixels ofall colors using the direction of resolution, as shown in FIG. 26B.Subsequently, as shown in FIG. 26C, the W pixels are replaced with Gpixels using a correlation between a W pixel and a G pixel. Thereafter,as shown in FIG. 26D, R and B pixels are generated for the RGB Bayerarrangement using a correlation between a W pixel and an R pixel and acorrelation between a W pixel and a B pixel.

The pixel addition process is described next with reference to schematicillustrations shown in FIGS. 27A to 27E. As shown in FIGS. 27A and 27B,for W, R, G, and B pixels, left-up and right-up diagonal FD addition isperformed. Thus, W, Cy (cyan), G, and Ye (yellow) colors are generated.Thereafter, as shown in FIG. 27C, the Bayer arrangement is generated bycomputing B=W−Ye and R=W−Cy. At that time, although the S/N ratios ofthe B and R colors is degraded due to the subtraction operations, thereproducibility of the colors is increased. Thereafter, as shown in FIG.27D, the components of the W pixels are fitted into the R, G, and Bpixels. Thus, as shown in FIG. 27E, four pixels for the RGB Bayerarrangement are generated.

Color Coding According to Fourth Example of First Exemplary Embodiment

A color conversion process at a time of full scanning is described firstwith reference to schematic illustrations shown in FIGS. 28A to 28D. Ina color arrangement of a 4×4 pixel block according to the fourth exampleof the first exemplary embodiment shown in FIG. 28A, the components of Wpixels arranged in a checkerboard pattern are expanded into pixels ofall colors using the direction of resolution, as shown in FIG. 28B.Subsequently, as shown in FIG. 28C, the W pixels are replaced with Gpixels using a correlation between a W pixel and a G pixel. Thereafter,as shown in FIG. 28D, R and B pixels are generated for the RGB Bayerarrangement using a correlation between a W pixel and an R pixel and acorrelation between a W pixel and a B pixel.

The four types of pixel addition can be used. The first type of pixeladdition is described next with reference to schematic illustrationsshown in FIGS. 29A to 29F. As shown in FIGS. 29A and 29B, for R and Bpixels, addition is performed on alternate pixels arranged in the upperleft-bottom right diagonal directions and the upper right-bottom leftdiagonal directions. Thus, W, Cy, and Ye colors are generated.Thereafter, as shown in FIG. 29C, each of the Cy and Ye components isexpanded into all pixels. Subsequently, as shown in FIG. 29D, thefollowing equations G=Cy+Ye−W, B=W−Ye, and R=W−Cy are computed. As shownin FIG. 29E, the components of the W pixels are then fitted into the R,G, and B pixels. Thus, as shown in FIG. 29F, four pixels for the RGBBayer arrangement are generated.

The second type of pixel addition is described next with reference toschematic illustrations shown in FIGS. 30A to 30D. As shown in FIG. 30A,for R pixels and B pixels, addition is performed between two alternatepixels arranged in the upper left-bottom right diagonal directions andin the upper right-bottom left diagonal directions, respectively. For Gpixels, an average value of the G value located at the center and four Gpixels located immediately above and beneath the center pixel and to theleft and right of the center pixel is computed. For W pixels, as shownin FIG. 30B, diagonal FD addition is performed. As shown in FIG. 30C,the components of the W pixels are then fitted into the R, G, and Bpixels. Thus, as shown in FIG. 30D, four pixels for the RGB Bayerarrangement are generated.

The third type of pixel addition is described next with reference toschematic illustrations shown in FIGS. 31A to 31C. As shown in FIG. 31A,for W pixels, addition is performed between two diagonally adjacentpixels. For each of R, G, and B pixels, addition is performed betweentwo alternate pixels arranged in a diagonal direction. Accordingly, acolor arrangement as shown in FIG. 31B is obtained. Subsequently, thecomponents of the W pixels are then fitted into the R, G, and B pixels.Thus, as shown in FIG. 31C, four pixels for the RGB Bayer arrangementare generated.

The fourth type of pixel addition is described next with reference toschematic illustrations shown in FIGS. 32A to 32D. As shown in FIG. 32A,for each of W, R, G, and B pixels, diagonal two-pixel FD addition withone pixel therebetween is performed. Thereafter, by combining thesignals of the W pixels arranged in a two-row-based checkerboardpattern, a color arrangement including a W checkerboard pattern can beobtained, as shown in FIG. 32B. Subsequently, as in the processperformed at a time of full scanning, the W pixels are replaced with Gpixels using a correlation between a W pixel and a G pixel, as shown inFIG. 32C. Thereafter, as shown in FIG. 32D, R and B pixels are generatedfor the RGB Bayer arrangement using a correlation between a W pixel andan R pixel and a correlation between a W pixel and a B pixel.

Color Coding According to Fifth Example of First Exemplary Embodiment

A color conversion process at a time of full scanning is described firstwith reference to schematic illustrations shown in FIGS. 33A to 33D. Ina color arrangement of a 4×4 pixel block according to the fifth exampleof the first exemplary embodiment shown in FIG. 33A, the components of Wpixels arranged in a checkerboard pattern are expanded into pixels ofall colors using the direction of resolution, as shown in FIG. 33B.Subsequently, as shown in FIG. 33C, the components of each of R and Bpixels are expanded into all pixels using a correlation between a Wpixel and an R pixel and a correlation between a W pixel and a B pixel.Thereafter, the following equation: G=W−R−B is computed. The componentsof the W pixels are fitted into all pixels. Thus, as shown in FIG. 33D,four pixels for the RGB Bayer arrangement are generated.

The pixel addition is described next with reference to schematicillustrations shown in FIGS. 34A to 34E. As shown in FIG. 34A, thecentroid locations of R, G, and B pixels are aligned. The signals of Rpixels are summed. The signals of G pixels are summed, and the signalsof B pixels are summed Subsequently, as shown in FIG. 34B, G signals aregenerated by computing the following equation: G=W−R−B. Thereafter, asshown in FIG. 34C, diagonal direction four-pixel addition is performedfor the B pixels, and up-and-down left-and-right five-pixel addition isperformed for the R pixels. Thus, the R pixels and the B pixels aregenerated. Finally, as shown in FIG. 34D, four pixels for the RGB Bayerarrangement are generated.

Color Coding According to Sixth Example of First Exemplary Embodiment

A color conversion process 1 at a time of full scanning is describedfirst with reference to schematic illustrations shown in FIGS. 35A to35D. In a color arrangement of a 4×4 pixel block according to the sixthexample of the first exemplary embodiment shown in FIG. 35A, thecomponents of W pixels arranged in a checkerboard pattern are expandedinto pixels of all colors using the direction of resolution, as shown inFIG. 35B. Subsequently, as shown in FIG. 35C, the W pixels are replacedwith G pixels using a correlation between a W pixel and a G pixel.Thereafter, as shown in FIG. 35D, R and B pixels are generated for theRGB Bayer arrangement using a correlation between a W pixel and an Rpixel and a correlation between a W pixel and a B pixel.

A color conversion process 2 at a time of full scanning is describednext with reference to schematic illustrations shown in FIGS. 36A to36D. Like the color conversion process 1, in a color arrangement of a4×4 pixel block according to the sixth example of the first exemplaryembodiment, the components of W pixels arranged in a checkerboardpattern are expanded into pixels of all colors using the direction ofresolution, as shown in FIG. 36A. Subsequently, as shown in FIG. 36B, Rand B pixels are generated using a correlation between a W pixel and anR pixel and a correlation between a W pixel and a B pixel. Thereafter,as shown in FIG. 36C, the signals of two G pixels adjacent to the Wpixel are added to the signal of the W pixel so as to be approximated toG (=W+2G). In this way, as shown in FIG. 36D, each of the R and B pixelsis generated for the Bayer arrangement. At that time, if the directionis found, the addition ratio can be dynamically changed.

The pixel addition process 1 is described next with reference to FIGS.37A to 37D. As shown in FIGS. 37A and 37B, left-up and right-up diagonalFD addition is performed for a W pixel and each of R, G, and B pixels.Thus, W, R, G, and B pixels are generated. Thereafter, as shown in FIG.37C, the components of the W pixel are fitted into each of the pixels.In this way, as shown in FIG. 37D, four pixels for the RGB Bayerarrangement are generated.

A pixel addition process 2 is described next with reference to FIGS. 38Ato 38D. As shown in FIGS. 38A and 38B, left-up and right-up diagonal FDaddition is performed for a W pixel and each of R, G, and B pixels.Thus, W, R, G, and B pixels are generated. Thereafter, as shown in FIG.38C, the components of the W pixel are fitted into each of the R and Bpixels. In this way, as shown in FIG. 38D, four pixels for the RGB Bayerarrangement are generated.

Color Coding According to Seventh Example of First Exemplary Embodiment

A color conversion process at a time of full scanning is described firstwith reference to schematic illustrations shown in FIGS. 39A to 39D. Ina color arrangement of a 4×4 pixel block according to the seventhexample of the first exemplary embodiment shown in FIG. 39A, thecomponents of G pixels arranged in a checkerboard pattern are expandedinto R and B pixels using the direction of resolution, as shown in FIG.39B. Subsequently, as shown in FIG. 39C, the W pixels are replaced withthe R and B pixels using a correlation between a W pixel and an R pixeland a correlation between a W pixel and a B pixel. Thus, as shown inFIG. 39D, four pixels for the RGB Bayer arrangement are generated.

A pixel addition process is described next with reference to schematicillustrations shown in FIGS. 40A to 40E. As shown in FIG. 40A, for Rpixels and B pixels, addition is performed between two alternate pixelsarranged in the upper left-bottom right diagonal directions and in theupper right-bottom left diagonal directions, respectively. Thus, asshown in FIG. 40B, 2R and 2B pixels are generated. Thereafter, as shownin FIG. 40C, diagonal diamond-shape addition is performed on G pixels.Thus, as shown in FIG. 40D, 4G pixels are generated. Subsequently, Gw,Rw, and Bw are added to the W pixels using a correlation between a Wpixel and an R pixel, a correlation between a W pixel and a B pixel, anda correlation between a W pixel and a B pixel. In this way, as shown inFIG. 40E, four pixels for the RGB Bayer arrangement are generated.

Color Coding According to Eighth Example of First Exemplary Embodiment

A color conversion process at a time of full scanning is described withreference to schematic illustrations shown in FIGS. 41A to 41D. In acolor arrangement of a 4×4 pixel block according to the eighth exampleof the first exemplary embodiment shown in FIG. 41A, the component of Wpixels are expanded into the G and W pixels so as to be arranged in acheckerboard pattern using the direction of resolution, as shown in FIG.41B. Subsequently, as shown in FIG. 41C, the components of the W pixelsare fitted into the G pixels. Thus, as shown in FIG. 41D, four pixelsfor the RGB Bayer arrangement are generated.

The pixel addition process is described next with reference to schematicillustrations shown in FIGS. 42A to 42D. As shown in FIG. 42A, thecentroid locations of G and W pixels are aligned. The signals of Gpixels are summed, and the signals of W pixels are summed Subsequently,as shown in FIG. 42B, the components of the W pixels are fitted into theG pixels. Thus, G=G+Gw is generated. Thereafter, as shown in FIG. 42C,diagonal direction four-pixel addition is performed for the B pixels,and up-and-down left-and-right five-pixel addition is performed for theR pixels. Thus, as shown in FIG. 42D, four pixels for the RGB Bayerarrangement are generated.

Color Coding According to Ninth Example of First Exemplary Embodiment

A color conversion process at a time of full scanning is described withreference to schematic illustrations shown in FIGS. 43A to 43C. In acolor arrangement of a 4×4 pixel block according to the ninth example ofthe first exemplary embodiment shown in FIG. 43A, the components of theG pixels are expanded into R and B pixels using the direction ofresolution, as shown in FIG. 43B. Subsequently, as shown in FIG. 43C,four pixels for the RGB Bayer arrangement are generated using acorrelation between a G pixel and an R pixel and a correlation between aG pixel and a B pixel.

The pixel addition process is described next with reference to schematicillustrations shown in FIGS. 44A and 44B. As shown in FIG. 44A, FDaddition is performed on the signals of four pixels of a 2×2 pixel blockof the same color. Finally, as shown in FIG. 44B, four pixels for theRGB Bayer arrangement are generated.

2. Second Exemplary Embodiment

A second exemplary embodiment is described next with reference to theaccompanying drawings.

System Configuration

FIG. 1 is a schematic illustration of an exemplary system configurationof a CMOS image sensor, which is an example of a solid-state imagingdevice (an X-Y addressing solid-state imaging device) according to asecond exemplary embodiment based on the first exemplary embodiment.

As shown in FIG. 1, each of CMOS image sensors 10, 10A, 10B, . . .includes a semiconductor substrate (hereinafter also referred to as a“sensor chip”) 11, a pixel array unit 12, a vertical drive unit 13, acolumn processing unit 14, a horizontal drive unit 15, a conversionprocessing unit 16, a system control unit 17, and a color filter array(a color filter unit) 30. As shown in FIG. 2, a unit pixel 20 includes aphotodiode 21, a transfer transistor 22, a reset transistor 23, anamplifying transistor 24, and a selection transistor 25, and a floatingdiffusion (FD) unit 26. A reference numeral “40” denotes a DSP circuit.

According to the present embodiment, in the CMOS image sensor 10A, thepixel array unit 12 is formed on the semiconductor substrate (the sensorchip) 11. In addition, a peripheral circuit unit is integrated onto thesemiconductor substrate 11 having the pixel array unit 12 formedthereon. For example, the peripheral circuit unit includes the verticaldrive unit 13, the column processing unit 14, the horizontal drive unit15, the conversion processing unit 16, and the system control unit 17.

The pixel array unit 12 includes a plurality of unit pixels (not shown),each including a photoelectric conversion element, two-dimensionallyarranged in an array. The unit pixel (hereinafter also simply referredto as a “pixel”) photoelectrically converts visible light incidentthereon into electrical charge in accordance with the intensity of thevisible light. The color filter array 30 is provided on the pixel arrayunit 12 on the side of a light receiving surface (a light incidentsurface). One of the key features of the present exemplary embodiment isthe color coding of the color filter array 30. The color coding of thecolor filter array 30 is described in more detail below.

Furthermore, in the pixel array unit 12, a pixel drive line 18 isdisposed in the left-right direction of FIG. 1 (a direction in which thepixels of a pixel row are arranged or the horizontal direction) for eachof the rows of the pixel array. Similarly, a vertical signal line 19 isdisposed in the up-down direction of FIG. 1 (a direction in which thepixels of a pixel column are arranged or the vertical direction) foreach of the columns of the pixel array. In FIG. 1, while only one pixeldrive line 18 is illustrated, the number of the pixel drive lines 18 isnot limited to 1. One end of the pixel drive line 18 is connected to anoutput terminal corresponding to one of the rows of the vertical driveunit 13.

For example, the vertical drive unit 13 includes a shift register and anaddress decoder. Although the detailed configuration thereof is notshown in FIG. 1, the vertical drive unit 13 includes a readout scanningsystem and a sweeping scanning system. The readout scanning systemsequentially scans the unit pixels from which signals are read by arow-by-row basis.

In contrast, prior to the readout scanning operation of the readout rowperformed by the readout scanning system by the time determined by ashutter speed, the sweeping scanning system performs sweeping scanningso that unnecessary electrical charge is swept (reset) out of thephotoelectric conversion elements of the unit pixels in the readout row.By sweeping (resetting) the unnecessary electrical charge using thesweeping scanning system, a so-called electronic shutter operation isperformed. That is, in the electronic shutter operation, the lightelectrical charge of the photoelectric conversion element is discarded,and a new exposure operation (accumulation of light electrical charge)is started.

A signal read through a readout operation performed by the readoutscanning system corresponds to the amount of light made incident afterthe immediately previous readout operation or electronic shutteroperation is performed. In addition, a period of time from a readouttime point of the immediately previous readout operation or a sweepingtime point of the electronic shutter operation to the readout time pointof the current readout operation corresponds to an accumulation time (anexposure time) of the light electrical charge in the unit pixel.

A signal output from each of the unit pixels in the pixel row selectedand scanned by the vertical drive unit 13 is supplied to the columnprocessing unit 14 via the corresponding one of the vertical signallines 19. For each of the pixel columns of the pixel array unit 12, thecolumn processing unit 14 performs predetermined signal processing onthe analog pixel signal output from the pixel in the selected row.

An example of the signal processing performed by the column processingunit 14 is a correlated double sampling (CDS) process. In the CDSprocess, the reset level and the signal level output from each of thepixels in the selected row are retrieved, and the difference between thelevels is computed. Thus, a signal of the pixels in the selected row isobtained. In addition, fixed pattern noise of the pixels is removed. Thecolumn processing unit 14 may have an analog-to-digital (A/D) conversionfunction for converting the analog pixel signal into a digital format.

For example, the horizontal drive unit 15 includes a shift register andan address decoder. The horizontal drive unit 15 sequentially selectsand scans a circuit portion corresponding to a pixel column of thecolumn processing unit 14. Each of the pixel columns is sequentiallyprocessed by the column processing unit 14 through the selectionscanning operation performed by the horizontal drive unit 15 and issequentially output.

The conversion processing unit 16 performs computation and convertssignals corresponding to the color arrangement of the color filter array(the color filter unit) 30 and output from the pixels of the pixel arrayunit 12 via the column processing unit 14 into signals corresponding tothe Bayer arrangement. Another of the key features of the presentembodiment is that the conversion processing unit 16 is mounted on thesubstrate on which the pixel array unit 12 is formed, that is, thesensor chip 11, a color conversion process is performed in the sensorchip 11, and a signal corresponding to the Bayer arrangement is outputfrom the sensor chip 11. The color conversion process performed by theconversion processing unit 16 is described in more detail below.

As widely used, the term “Bayer arrangement” represents a colorarrangement in which a color serving as a primary color informationcomponent of a luminance signal for high resolution is arranged in acheckerboard pattern, and the other two colors serving as colorinformation components of the luminance signal for not-so-highresolution are arranged in the other area of the checkerboard pattern.In a basic color coding form of the Bayer arrangement, green (G) thathas high contribution of the luminance signal is arranged in acheckerboard pattern, and red (R) and blue (B) are arranged in the otherarea of the checkerboard pattern.

The system control unit 17 receives a clock provided from outside thesensor chip 11 and data for indicating an operating mode. In addition,the system control unit 17 outputs data representing internalinformation of the CMOS image sensor 10. Furthermore, the system controlunit 17 includes a timing generator that generates a variety of timingsignals. The system control unit 17 controls driving of the verticaldrive unit 13, the column processing unit 14, the horizontal drive unit15, and the conversion processing unit 16 using the variety of timingsignals generated by the timing generator.

Circuit Configuration of Unit Pixel

FIG. 2 is an exemplary circuit diagram of a unit pixel 20. As shown inFIG. 2, the unit pixel 20 illustrated in the exemplary circuit diagramincludes a photoelectric conversion element (e.g., a photodiode 21) andfour transistors (e.g., a transfer transistor 22, a reset transistor 23,an amplifying transistor 24, and a selection transistor 25).

In this example, N-channel MOS transistors are used for the transfertransistor 22, the reset transistor 23, the amplifying transistor 24,and the selection transistor 25. However, the combination of aconductive type using the transfer transistor 22, the reset transistor23, the amplifying transistor 24, and the selection transistor 25 isonly an example, and the combination is not limited thereto.

For example, as the pixel drive line 18, three drive lines, that is, atransfer line 181, a reset line 182, and a selection line 183 areprovided to each of the unit pixels 20 in the same pixel row. One end ofthe transfer line 181, one end of the reset line 182, and one end of theselection line 183 are connected to an output terminal corresponding toone of the pixel rows of the vertical drive unit 13.

An anode electrode of the photodiode 21 is connected to a negative powersupply (e.g., the ground). The photodiode 21 photoelectrically convertsreceived light into photocharges (photoelectrons in this exemplaryembodiment) in accordance with the amount of received light. A cathodeelectrode of the photodiode 21 is connected to the gate electrode of theamplifying transistor 24 via the transfer transistor 22. A node 26electrically connected to the gate electrode of the amplifyingtransistor 24 is referred to as a “floating diffusion (FD) unit”.

The transfer transistor 22 is connected between the cathode electrode ofthe photodiode 21 and the FD unit 26. When a transfer pulse φTRF havingan active high level (e.g., a Vdd level) (hereinafter referred to as a“high active transfer pulse”) is supplied to a gate electrode of thetransfer transistor 22 via the transfer line 181, the transfertransistor 22 is turned on. Thus, the transfer transistor 22 transfersthe photocharges photoelectrically converted by the photodiode 21 to theFD unit 26.

A drain electrode of the reset transistor 23 is connected to a pixelpower supply Vdd. The source electrode of the reset transistor 23 isconnected to the FD unit 26. Before the signal charge is transferredfrom the photodiode 21 to the FD unit 26, a high active reset pulse φRSTis supplied to a gate electrode of the reset transistor 23 via the resetline 182. When the reset pulse φRST is supplied to the reset transistor23, the reset transistor 23 is turned on. Thus, the reset transistor 23resets the FD unit 26 by discharging the electrical charge of the FDunit 26 to the pixel power supply Vdd.

The gate electrode of the amplifying transistor 24 is connected to theFD unit 26. A drain electrode of the amplifying transistor 24 isconnected to the pixel power supply Vdd. After the FD unit 26 is resetby the reset transistor 23, the amplifying transistor 24 outputs thepotential of the FD unit 26 in the form of a reset signal (a resetlevel) Vreset. In addition, after the signal charge is transferred bythe transfer transistor 22, the amplifying transistor 24 outputs thepotential of the FD unit 26 in the form of a light accumulation signal(a signal level) Vsig.

For example, a drain electrode of the selection transistor 25 isconnected to the source electrode of the amplifying transistor 24. Thesource electrode of the selection transistor 25 is connected to thevertical signal line 17. When a high active selection pulse φSEL issupplied to the gate electrode of the selection transistor 25 via theselection line 163, the selection transistor 25 is turned on. Thus, theselection transistor 25 causes the unit pixel 20 to enter a selectedmode so that a signal output from the amplifying transistor 24 isrelayed to the vertical signal line 17.

Note that a circuit configuration in which the selection transistor 25is connected between the pixel power supply Vdd and the drain of theamplifying transistor 24 may be employed.

It should be noted that the pixel structure of the unit pixel 20 is notlimited to the above-described four-transistor pixel structure. Forexample, the unit pixel 20 may have a three-transistor pixel structurein which the functions of the amplifying transistor 24 and the selectiontransistor 25 are performed by one transistor. Thus, any configurationof a pixel circuit can be employed.

In general, in order to increase a frame rate when a moving image iscaptured, pixel addition in which signals output from a plurality ofneighboring pixels are summed and read out is performed. The pixeladdition is performed in a pixel, signal lines, the column processingunit 14, or a downstream signal processing unit. In the presentembodiment, for example, a pixel structure in which four pixels arrangedso as to be adjacent to one another in the vertical direction and thehorizontal direction is described.

FIG. 3 is a circuit diagram of an exemplary configuration of a circuitthat allows pixel addition for four neighboring pixels to be performedin the pixels. The same numbering will be used in describing FIG. 3 aswas utilized above in describing FIG. 2, where appropriate.

In FIG. 3, the photodiodes 21 of the four pixels arranged so as to beadjacent to one another in the vertical direction and the horizontaldirection are denoted as photodiodes 21-1, 21-2, 21-3, and 21-4. Fourtransfer transistors 22-1, 22-2, 22-3, and 22-4 are provided to thephotodiodes 21-1, 21-2, 21-3, and 21-4, respectively. In addition, theone reset transistor 23, the one amplifier transistor 24, and the oneselection transistor 25 are used.

That is, one of the electrodes of the transfer transistor 22-1, one ofthe electrodes of the transfer transistor 22-2, one of the electrodes ofthe transfer transistor 22-3, and one of the electrodes of the transfertransistor 22-4 are connected to the cathode electrode of the photodiode21-1, the cathode electrode of the photodiode 21-2, the cathodeelectrode of the photodiode 21-3, and the cathode electrode of thephotodiode 21-4, respectively. The other electrode of the transfertransistor 22-1, the other electrode of the transfer transistor 22-2,the other electrode of the transfer transistor 22-3, and the otherelectrode of the transfer transistor 22-4 are commonly connected to thegate electrode of the amplifier transistor 24. In addition, the FD unit26 that is shared by the photodiodes 21-1, 21-2, 21-3, and 21-4 iselectrically connected to the gate electrode of the amplifier transistor24. The drain electrode of the reset transistor 23 is connected to thepixel power supply Vdd, and the source electrode of the reset transistor23 is connected to the FD unit 26.

In the above-described pixel structure that supports the pixel additionfor four neighboring pixels, by providing the transfer pulse φTRF to thefour transfer transistors 22-1, 22-2, 22-3, and 22-4 at the same time,pixel addition for four neighboring pixels can be performed. That is,the signal charges transferred from the photodiodes 21-1, 21-2, 21-3,and 21-4 to the FD unit 26 by the four transfer transistors 22-1, 22-2,22-3, and 22-4 are summed by the FD unit 26.

In contrast, by providing the transfer pulse φTRF to the four transfertransistors 22-1, 22-2, 22-3, and 22-4 at different points of time, thesignals can be output on a pixel-by-pixel basis. That is, when a movingimage is captured, the frame rate can be increased by performing pixeladdition. In contrast, when a still image is captured, the resolutioncan be increased by independently reading the signals of all of thepixels.

Color Coding of Color Filter Array

The color coding of the color filter array 30 which is one of thefeatures of the present exemplary embodiment is described next.

According to the present exemplary embodiment, the color filter array 30employs color coding in which filters of a first color serving as one ofprimary color information components of a luminance signal are arrangedin a checkerboard pattern. In addition, filters of a second color of theprimary color information components for a series of four pixels form agroup, and the groups are arranged so as to form a stripe pattern in oneof a diagonal direction, a vertical direction, and a horizontaldirection. In the present embodiment, the filters of a first color and asecond color of the primary colors of a luminance signal are, forexample, a W filter and a G filter.

The filters of W and G colors which are primary components of aluminance signal have sensitivities higher than those of filters ofother colors (more specifically, R and B filters). In particular, the Wfilter has sensitivity about twice that of the G filter. Accordingly, byarranging the W and G filters (in particular, the W filters) in acheckerboard pattern, the sensitivity (the S/N ratio) can be increased.However, since a W filter contains various color information, a falsecolor which is different from the original color of a subject tends toappear. In contrast, although a G filter has sensitivity lower than thatof a W filter, a G filter produces few false colors. That is, there is atrade-off between the sensitivity and generation of a false color.

When W filters are arranged in a checkerboard pattern for the primarycolor, R, G, and B filters are arranged in the other areas of thecheckerboard pattern for the other color information components. Incontrast, when G filters are arranged in a checkerboard pattern for theprimary color, R and B filters are arranged in the other areas of thecheckerboard pattern for the other color information components.

In this way, by using, for the color filter array 30, color coding inwhich W filters for the primary color of a luminance signal are arrangedin a checkerboard pattern, the sensitivity of the CMOS image sensor 10can be increased, since the W filter has a sensitivity higher than afilter of another color. In contrast, by using, for the color filterarray 30, color coding in which G filters for the primary color of aluminance signal are arranged in a checkerboard pattern, the colorreproducibility of the CMOS image sensor 10 can be increased, since theG filter produces few false colors.

In addition, in the CMOS image sensor 10A according to the presentembodiment, when the color filter array 30 using either one of the colorcoding methods is used, a signal corresponding to the color arrangementis converted into a signal corresponding to the Bayer arrangement by thesensor chip 11. At that time, since the color serving as the primarycomponent of a luminance signal is arranged in a checkerboard pattern,signals of other colors of pixels adjacent to the color in the verticaldirection and the horizontal direction can be restored using the signalof the color serving as the primary component of a luminance signal.Consequently, the efficiency of color conversion performed by theconversion processing unit 16 can be increased.

In addition, by outputting the signal corresponding to the Bayerarrangement from the sensor chip 11, an existing DSP for the Bayerarrangement can be used as a downstream signal processing unit.Basically, the DSP for the Bayer arrangement generates a luminancesignal Y and two color difference signals U(B−Y) and V(R−Y) using thesignal output from the sensor chip 11 and corresponding to the Bayerarrangement.

In this way, since an existing DSP for the Bayer arrangement can beused, development of a new DSP that is significantly costly is notnecessary even when the color coding of the color filter array 30 ischanged. Accordingly, a camera module including a DSP can be produced ata low cost. As a result, the widespread use of the color filter array 30using, in particular, a W filter can be expected.

Examples of Color Coding of Color Filter Array

Examples of color coding that facilitate conversion from a signalcorresponding to a color arrangement in which filters of a color servingas a primary component of a luminance signal are arranged in acheckerboard pattern to a signal corresponding to an RGB Bayerarrangement are described in detail below.

First Example of Second Exemplary Embodiment

FIG. 45 is a color arrangement diagram illustrating color codingaccording to a first example of the second exemplary embodiment. Asshown in FIG. 45, in the color coding according to the first example ofthe second exemplary embodiment, W filters are arranged in acheckerboard pattern. In addition, R filters are arranged in acheckerboard pattern at a two-pixel pitch in the vertical direction andthe horizontal direction. Similarly, B filters are arranged in acheckerboard pattern at a two-pixel pitch in the vertical direction andthe horizontal direction. Furthermore, one of the R filters isdiagonally shifted from one of the B filters by one pixel. In addition,G filters are arranged in the other area of the checkerboard pattern.

More specifically, in an array including four pixels in each of thevertical direction and the horizontal direction, W filters are arrangedin a checkerboard pattern. R filters are arranged in the second row andfirst column and in the fourth row and third column. B filters arearranged in the first row and second column and in the third row andfourth column. This array is the checkerboard pattern having a two-pixelpitch. In addition, G filters are arranged in the other area of thecheckerboard pattern. At that time, the G filters form a diagonal stripepattern.

Second Example of Second Exemplary Embodiment

FIG. 46 is a color arrangement diagram illustrating color codingaccording to a second example of the second exemplary embodiment. Asshown in FIG. 46, in the color coding according to the second example ofthe second exemplary embodiment, W filters are arranged in acheckerboard pattern. R filters are squarely arranged in a checkerboardpattern at a four-pixel pitch in the vertical direction and thehorizontal direction. Similarly, B filters are squarely arranged in acheckerboard pattern at a four-pixel pitch in the vertical direction andthe horizontal direction. In addition, the R filters and the B filtersare diagonally shifted from each other by two pixels. In addition, Gfilters are arranged in the other area in the checkerboard pattern.

More specifically, in an array including four pixels in each of thevertical direction and the horizontal direction, W filters are arrangedin a checkerboard pattern. R filters are disposed in the third row andfourth column. B filters are disposed in the first row and second columnand in the second row and first column. This array is the square arrayhaving a four-pixel pitch in the vertical direction and the horizontaldirection. In addition, G filters are arranged in the other area of thecheckerboard pattern. At that time, the G filters form a diagonal stripepattern.

The color coding methods according to the above-described first andsecond examples use a color arrangement in which W filters having acolor serving as a primary component of a luminance signal thatmaximizes the output level are arranged in a checkerboard pattern. Sincethe W filters that include R, G, and B color components are arranged ina checkerboard pattern, the conversion accuracy of a signalcorresponding to the RGB Bayer arrangement can be increased.

The key feature of these color coding methods is that if the W filtersare replaced with G filters during the color conversion processdescribed below, the locations of the R and B filters are partiallycoincident with those of R and B filters of the Bayer arrangement. Inaddition, for the locations at which colors are not coincident,information regarding pixels of the W filters can be used. Thus, R and Bpixel information can be restored. As a result, conversion efficiencycan be significantly increased.

In addition, the key feature of the color coding methods according tothe first and second examples of the second exemplary embodiment is thatthe W filters are arranged in a checkerboard pattern, and a series offour G filters arranged in a diagonal direction repeatedly appears sothat a diagonal stripe pattern is formed. In such color coding methods,by adding the signal of a pixel of a G filter adjacent to a pixel of a Wfilter to the signal of the W filter and using the sum as a primarycomponent of a luminance signal, the intensity of the luminance signalcan be increased. Accordingly, the sensitivity (the S/N ratio) can beincreased. In particular, in the color coding method according to thefirst example, the R filters are arranged in a checkerboard pattern at atwo-pixel pitch in the vertical direction and in the horizontaldirection. In addition, each of the R filters is diagonally shifted fromone of the B filters by one pixel. Accordingly, the efficiency ofconversion into a signal corresponding to the Bayer arrangement can beincreased.

In the color coding methods according to the first and second examplesof the second embodiment, a series of four G filters arranged in adiagonal direction repeatedly appears so that a diagonal stripe patternis formed. Accordingly, in such color coding methods, by adding thesignal of a G pixel or the signals of G pixels adjacent to a W pixel tothe signal of the W pixel and using the sum signal as a primarycomponent of a luminance signal, a high sensitivity (a high S/N ratio)can be provided without a decrease in resolution. This advantage can beprovided not only when a series of four G filters arranged in a diagonaldirection repeatedly appears so that a diagonal stripe pattern isformed, but also when a series of four G filters arranged in thevertical direction or the horizontal direction repeatedly appears sothat a stripe pattern is formed.

Sensitivity Ratio W:G:R:B

A sensitivity ratio W:G:R:B is described next. In color coding includingW filters, a W filter pixel that has a high output signal level issaturated earlier than a pixel of other color filters. Accordingly, itis necessary that a sensitivity balance among W, G, R, and B pixels besustained, that is, the sensitivity ratio W:G:R:B be adjusted bydecreasing the sensitivity of the W filter pixel and increasing thesensitivities of the other color filter pixel relative to thesensitivity of the W filter pixel.

In order to adjusting the sensitivity ratio, widely used exposurecontrol techniques can be used. More specifically, by adjusting the sizeof an on-chip microlens provided outside the color filter array 30 foreach of the pixels of the color filter array 30, the balance among theamounts of light made incident on individual colors is sustainable(refer to, for example, Japanese Unexamined Patent ApplicationPublication No. 9-116127). By using this technique and decreasing thesize of an on-chip microlens for a W color than that for one of theother colors, the sensitivity of the W filter pixel can be decreased. Inthis way, the sensitivity of one of the other color filter pixel can berelatively increased.

Alternatively, an exposure light control technique in which, in colorcoding including a W filter, the difference between the sensitivitiescan be reduced by removing an on-chip microlens for a W pixel, and acolor S/N can be improved by increasing the color sensitivity can beused (refer to, for example, Japanese Unexamined Patent ApplicationPublication No. 2007-287891). By using this technique, the sensitivityof the W filter pixel can be decreased. In this way, the sensitivity ofone of the other color filter pixel can be relatively increased.

Still alternatively, a technique for preventing the improper colorbalance by performing shutter exposure control in which the exposuretime of a G filter pixel is decreased compared with that for an R or Bfilter pixel can be used (refer to, for example, Japanese UnexaminedPatent Application Publication No. 2003-60992). By combining such ashutter exposure control technique with control of light-receiving area,the sensitivity of the W filter pixel can be decreased. In this way, thesensitivity of one of the other color filter pixel can be relativelyincreased. Furthermore, in particular, the occurrence of a coloredoutline of a moving subject can be eliminated. As a result, anachromatizing process performed by an external signal processing unit (aDSP) is not necessary.

Note that the above-described three exposure control techniques used foradjusting the sensitivity ratio are only examples. The exposure controltechnique is not limited thereto.

For example, the size of the on-chip microlens for a W pixel is set insuch a manner that the output levels of the W, G, B, and R pixels aresubstantially in the proportion 2:1:0.5:0.5.

For 1.1-μm pixels, when the size of an on-chip microlens for a W pixelis varied by ±0.1 μm, the area is doubled and halved. Therefore, thesensitivity is doubled or halved. Accordingly, the output level of a Wpixel having an area the same as that of a G pixel and having an outputlevel double that of the G pixel can be adjusted so that the outputlevels are the same. Even when the size of the on-chip microlens isvaried by ±0.05 μm, the area is ±1.42 times the original area and,therefore, the sensitivity of the W pixel can be reduced to 1.42 timesthat of the G pixel. In such a case, the further adjustment of thesensitivity may be performed by shutter exposure control.

Color Conversion Process

A process for converting the signals into signals corresponding to theRGB Bayer arrangement (i.e., a color conversion process) performed bythe conversion processing unit 16 is described in more detail next.

The following two types of color conversion process are provided: acolor conversion process performed when a still image is captured (at atime of full scanning in which all of the pixels are scanned) and acolor conversion process performed when a moving image is captured (at atime of pixel addition in which signals of a plurality of pixelsneighboring a pixel is added to the signal of the pixel). In the case ofcolor coding according to the first and second examples, a colorconversion process with a high sensitivity can be performed.Accordingly, a low luminance mode can be used and, therefore, the colorconversion process performed at a time of full scanning can be dividedinto two color conversion processes.

One of the two color conversion processes is performed when theluminance of incident light is higher than a predetermined referenceluminance. This color conversion process is referred to as a “colorconversion process 1”. The other color conversion process is performedwhen the luminance of incident light is lower than or equal to thepredetermined reference luminance. This color conversion process isreferred to as a “color conversion process 2”. In addition, the colorconversion process performed at a time of pixel addition can be dividedinto a plurality of color conversion processes in accordance with thecombinations of the pixels to be added.

Color Coding According to First Example of Second Exemplary Embodiment

A color conversion process performed for the color coding according tothe first example of the second exemplary embodiment is described next.First, the color conversion process 1 performed in a high luminance modeat a time of full scanning is described with reference to a flowchartshown in FIG. 13 and schematic illustrations shown in FIGS. 14A to 14D.

As illustrated by the flowchart shown in FIG. 13, the color conversionprocess 1 in a high luminance mode is realized by sequentiallyperforming the processing in steps S11, S12, and S13. FIG. 14Aillustrates a 4×4 color arrangement of the color coding according to thefirst example.

In step S11, as shown in FIG. 14B, the components of white (W) pixelsarranged in a checkerboard pattern are expanded into pixels of allcolors by determining the direction of resolution. As used herein, theterm “direction of resolution” refers to a direction in which pixelsignals are present. In FIG. 14B, “W” surrounded by a transparent squarerepresents a component of a W pixel after the component of the W pixelis expanded into each of all colors.

In order to expand a component of a W pixel into pixels of other colors,signal processing based on a widely used directional correlation can beapplied. For example, in signal processing based on a directionalcorrelation, a plurality of color signals corresponding to a given pixelis acquired, and the correlation value in the vertical direction and/orthe horizontal direction is obtained (refer to, for example, JapanesePatent No. 2931520).

In step S12, as shown in FIG. 14C, a W pixel is replaced with a G pixelusing a correlation between a W pixel and a G pixel. As can be seen fromthe above-described color arrangements of various color coding, a Wpixel is adjacent to a G pixel. In terms of correlation between a Wpixel and a G pixel in a certain area, the W pixel and the G pixel havea strong correlation since either one of the W pixel and the G pixel hasa color serving as a primary component of a luminance signal. Thus, thecorrelation value (correlation coefficient) is nearly 1. By determiningthe direction of resolution using the color correlation and changing theoutput level of a W pixel to the level equivalent to the output level ofa G pixel, the W pixel is replaced with the G pixel.

In step S13, an R pixel and a B pixel are generated for the Bayerarrangement shown in FIG. 14D using correlation between the W pixel andthe R pixel and correlation between the W pixel and the B pixel. Since aW pixel includes R, G, and B color components, the correlation betweenthe W pixel and the R pixel and the correlation between the W pixel andthe B pixel can be obtained. For the signal processing, an existingtechnique in which a luminance signal to be replaced with G in afour-color arrangement is generated for every pixel by interpolation canbe used (refer to, for example, Japanese Unexamined Patent ApplicationPublication No. 2005-160044).

Subsequently, the color conversion process 2 performed in a lowluminance mode at a time of full scanning is described with reference toa flowchart shown in FIG. 15 and schematic illustrations shown in FIGS.16A to 16D.

First, as shown in FIG. 16A, the direction of resolution is examined byusing signal processing based on the above-described directionalcorrelation (step S21). Thereafter, it is determined whether thedirection of resolution can be determined (step S22). If the directionof resolution can be determined, the components of the W pixels arrangedin a checkerboard pattern are expanded into pixels of all colors (stepS23).

Subsequently, as shown in FIG. 16B, R pixels and B pixels are generatedusing the correlation between a W pixel and an R pixel and thecorrelation between a W pixel and a B pixel (step S24). Thereafter, asshown in FIG. 16C, the signals of two R pixels adjacent to the W pixelare added to the signal of the W pixel so as to be approximated to G(=W+2G). In this way, as shown in FIG. 16D, each of the R and B pixelsis generated for the Bayer arrangement (step S25).

However, if, in step S22, the direction of resolution is not determined,each of the R and B pixels is generated for the Bayer arrangement byusing simple four-pixel averaging interpolation using four pixelsadjacent to the W pixel in the vertical direction and the horizontaldirection (step S26).

As described above, by using the color conversion process 1 and colorconversion process 2 in accordance with the luminance of the incidentlight, signals corresponding to a color arrangement in which W filtersare arranged in a checkerboard pattern can be converted into signalscorresponding to the RGB Bayer arrangement on the sensor chip 11, andthe signals can be output.

Two color conversion processes performed at a time of pixel addition formoving image capturing are described next. Hereinafter, one of the twocolor conversion processes is referred to as a “pixel addition process1”, and the other color conversion process is referred to as a “pixeladdition process 2”.

First, the pixel addition process 1 is described with reference to aflowchart shown in FIG. 47 and schematic illustrations shown in FIGS.48A to 48D.

Addition is performed on two W pixels diagonally adjacent to each other(step S31). More specifically, as shown in FIG. 48A, addition isperformed on a pixel of interest and a pixel located to the lower rightof the pixel of interest (i.e., a pixel to the right by one column andbelow by one row). This addition can be performed in the pixel structureshown in FIG. 3 by providing the transfer pulse clITRF to the transfertransistors 22 of the two pixels which are the targets of addition (thetransfer transistors 22-1 and 22-4 in this example) at the same time. Inthis way, two-pixel addition can be performed in the FD unit 26.Hereinafter, such pixel addition is referred to as “FD addition”.

Subsequently, for each of R, G, and B pixels, addition is performed ontwo pixels arranged in the opposite diagonal direction and having onepixel therebetween (step S32). More specifically, as shown in FIG. 48B,addition is performed on a pixel of interest and a pixel located to thelower left of the pixel of interest with one pixel there between (i.e.,a pixel to the right by two columns and below by two rows). For each ofR, G, and B pixels, when the column processing unit 14 shown in FIG. 1has an A/D conversion function, this addition can be performed duringA/D conversion.

More specifically, in the color arrangement shown in FIG. 19, thesignals of pixels B1 and G1 are read independently. After the signalsare A/D-converted, the signals of pixels B2 and G3 are continuously readand A/D-converted. In this way, two-pixel addition can be performed. Inorder to perform pixel addition during A/D conversion performed by thecolumn processing unit 14, an existing technique for converting ananalog pixel signal into a digital pixel signal using a counter can beused (refer to, for example, Japanese Unexamined Patent ApplicationPublication No. 2006-033454).

Hereinafter, this pixel addition performed using a counter of an A/Dconverter is referred to as “counter addition”. When counter addition isperformed and if the gain is changed on a line-by-line basis, theaddition ratio can be varied. Similarly, counter addition can beperformed on R pixels. Note that, in the above-described two-pixeladdition for the W pixels, FD addition is performed between the pixelsW1 and W2 and between the pixels W3 and W4.

Thereafter, as shown in FIG. 48C, the components of the W pixel arefitted into the R, G, and B pixels (step S33). Subsequently, as shown inFIG. 48D, four pixels for the RGB Bayer arrangement are generated (stepS34).

Subsequently, the pixel addition process 2 is described with referenceto a flowchart shown in FIG. 20 and schematic illustrations shown inFIGS. 21A to 21D.

First, for W pixels and G pixels, FD addition is performed between twopixels arranged in the upper left-bottom right diagonal directions andthe upper right-bottom left diagonal directions, respectively. Thus, W,R, G, and B pixels are generated (step S41). More specifically, for theW pixels, as shown in FIG. 21A, FD addition is performed between a pixelof interest and a pixel located at the lower right of the pixel ofinterest (i.e., a pixel to the right by one column and below by onerow). For the G pixels, as shown in FIG. 21B, FD addition is performedbetween a pixel of interest and a pixel located at the lower left of thepixel of interest (i.e., a pixel to the left by one column and below byone row).

Note that, in eight pixels of a 4×4 block, a pair of R and B signals isnot used. That is, R pixels and B pixels are read in a thinning-outmanner without performing pixel addition. Accordingly, the sensitivityof R and B is decreased, as compared with that in the pixel additionprocess 1. Consequently, a color S/N ratio is low.

Thereafter, as shown in FIG. 21C, the components of the W pixel arefitted into the R, G, and B pixels (step S42). Subsequently, as shown inFIG. 21D, four pixels for the RGB Bayer arrangement are generated (stepS43). In the case of the pixel addition process 2, the centroid locationof the Bayer arrangement is slightly shifted from that in the pixeladdition process 1.

As described above, by using one of the pixel addition process 1 and thepixel addition process 2 when a moving image is captured, signalscorresponding to a color arrangement in which W filters are arranged ina checkerboard pattern can be converted into signals corresponding tothe RGB Bayer arrangement on the sensor chip 11, and the signals can beoutput.

Color Coding According to Second Example of Second Exemplary Embodiment

A color conversion process for the color coding according to the secondexample of the second exemplary embodiment is described next. A seriesof processes for the color coding according to the second example of thesecond exemplary embodiment is based on that for the color codingaccording to the first example of the second exemplary embodiment.

A color conversion process 1 at a time of full scanning is describedwith reference to schematic illustrations shown in FIGS. 49A to 49D. Ina color arrangement of a 4×4 pixel block according to the second exampleshown in FIG. 49A, the components of W pixels arranged in a checkerboardpattern are expanded into pixels of all colors using the direction ofresolution, as shown in FIG. 49B. Subsequently, as shown in FIG. 49C,the W pixels are replaced with G pixels using a correlation between a Wpixel and a G pixel. Thereafter, as shown in FIG. 49D, R and B pixelsare generated for the RGB Bayer arrangement using a correlation betweena W pixel and an R pixel and a correlation between a W pixel and a Bpixel.

A color conversion process 2 at a time of full scanning is describednext with reference to schematic illustrations shown in FIGS. 50A to50D. As in the color conversion process 1, in a color arrangement of a4×4 pixel block according to the second example, the components of Wpixels arranged in a checkerboard pattern are expanded into pixels ofall colors using the direction of resolution, as shown in FIG. 50A.Subsequently, as shown in FIG. 50B, R pixels and B pixels are generatedusing a correlation between a W pixel and an R pixel and a correlationbetween a W pixel and a B pixel. Thereafter, as shown in FIG. 50C, thesignals of two G pixels adjacent to the W pixel are added to the signalof the W pixel so as to be approximated to G (=W+2G). In this way, asshown in FIG. 50D, each of the R and B pixels is generated for the Bayerarrangement. At that time, if the direction is found, the addition ratiocan be dynamically changed.

The pixel addition process 1 is described next with reference to FIGS.51A to 51D. As shown in FIGS. 51A and 51B, left-up and right-up diagonalFD addition is performed for a W pixel and each of R, G, and B pixels.Thus, W, R, G, and B pixels are generated. Thereafter, as shown in FIG.51C, the components of the W pixel are fitted into each of the pixels.In this way, as shown in FIG. 51D, four pixels for the RGB Bayerarrangement are generated.

A color conversion process 2 is described next with reference to FIGS.52A to 52D. As shown in FIGS. 52A and 52B, left-up and right-up diagonalFD addition is performed for a W pixel and each of R, G, and B pixels.Thus, W, R, G, and B pixels are generated. Thereafter, as shown in FIG.51C, the components of the W pixel are fitted into each of the R and Bpixels. In this way, as shown in FIG. 52D, four pixels for the RGB Bayerarrangement are generated.

As described above, by performing the above-described signal processingfor the color coding according to the first and second examples in whichW filters having a color serving as a primary component of a luminancesignal are arranged in a checkerboard pattern and G filters arediagonally arranged so as to form a stripe pattern, the followingadvantage can be provided. That is, by adding the signal of a G pixeladjacent to a W pixel to the signal of the W pixel and performing signalprocessing using the sum as a primary component of the luminance signal,the intensity of the luminance signal can be increased. Accordingly, thesensitivity can be increased with a minimal decrease in resolution.

Third Exemplary Embodiment

System Configuration

FIG. 53 is a schematic illustration of an exemplary system configurationof a CMOS image sensor, which is an example of a solid-state imagingdevice (an X-Y addressing solid-state imaging device) according to athird exemplary embodiment. The same numbering will be used indescribing FIG. 53 as was utilized above in describing FIG. 1, whereappropriate.

In the above-described CMOS image sensor 10A according to the secondexemplary embodiment, the conversion processing unit 16 disposed on thesensor chip 11 converts signals corresponding to the color arrangementof the color filter array 30 into signals corresponding to the RGB Bayerarrangement. In contrast, in a CMOS image sensor 10B according to thepresent embodiment, W, R, G, and B signals corresponding to the colorarrangement of the color filter array 30 are directly output from thesensor chip 11 in the form of raw data.

In addition, according to the present embodiment, the CMOS image sensor10B allows a DSP circuit 40, which is an external circuit of the sensorchip 11, to perform a color conversion process on the raw data outputfrom the sensor chip 11. The DSP circuit 40 converts the W, R, G, and Bsignals output from the sensor chip 11 into signals corresponding to theRGB Bayer arrangement. Thereafter, the DSP circuit 40 performs a widelyused demosaic process. In the demosaic process, color information isadded to the signal of each of the pixels that has only monochromeinformation by supplying missing color information using the signals ofpixels surrounding the pixel, and a full-color image is generated.

In this way, the key feature of the CMOS image sensor 10B according tothe present embodiment is that the W, R, G, and B signals correspondingto the color arrangement of the color filter array 30 are directlyoutput from the sensor chip 11 and are converted into the signalscorresponding to the RGB Bayer arrangement by the DSP circuit 40.Accordingly, since the configurations and operations of the pixel arrayunit 12, the vertical drive unit 13, the column processing unit 14, thehorizontal drive unit 15, and the system control unit 17 are similar tothose of the second exemplary embodiment, the descriptions thereof arenot repeated.

Color Coding of Color Filter Array

Like the CMOS image sensor 10A according to the second exemplaryembodiment, the CMOS image sensor 10B according to the presentembodiment has color coding of the color filter array 30 thatfacilitates conversion into signals corresponding to the RGB Bayerarrangement.

That is, the color filter array 30 employs color coding in which filtersof a first color (W or G) serving as one of primary color informationcomponents of a luminance signal are arranged in a checkerboard pattern.In addition, filters of a second color (G or W) of the primary colorinformation components for a series of four pixels form a group, and thegroups are arranged so as to form a stripe pattern in one of a diagonaldirection, a vertical direction, and a horizontal direction. Theadvantage of the use of the color filter array 30 having such colorcoding is the same as that of the second exemplary embodiment.

Examples of Color Coding of Color Filter Array

Like the second exemplary embodiment, according to the third exemplaryembodiment, first and second examples of color coding can be provided.The first and second examples facilitate conversion from a signalcorresponding to a color arrangement in which filters of a color servingas a primary component of a luminance signal are arranged in acheckerboard pattern into a signal corresponding to an RGB Bayerarrangement.

That is, in the color coding according to the first example of the thirdexemplary embodiment, W filters are arranged in a checkerboard pattern.In addition, R filters are arranged in a checkerboard pattern at atwo-pixel pitch in the vertical direction and the horizontal direction.Similarly, B filters are arranged in a checkerboard pattern at atwo-pixel pitch in the vertical direction and the horizontal direction.Furthermore, each of the R filters is diagonally shifted from one of theB filters by one pixel. In addition, G filters are arranged in the otherarea of the checkerboard pattern (refer to FIG. 45). Still furthermore,in the color coding according to the second example of the thirdexemplary embodiment, W filters are arranged in a checkerboard pattern.R filters are squarely arranged at a four-pixel pitch in the verticaldirection and the horizontal direction. Similarly, B filters aresquarely arranged at a four-pixel pitch in the vertical direction andthe horizontal direction. In addition, two of the R filters and two ofthe B filters are alternately arranged in a diagonal direction. Inaddition, G filters are arranged in the other area in the checkerboardpattern (refer to FIG. 46).

Color Conversion Process

A process performed by the DSP circuit 40 for converting signals thatcorrespond to the color arrangement of the color filter array 30 andthat are output from the sensor chip 11 in the form of raw data intosignals corresponding to the RGB Bayer arrangement is described in moredetail next.

Like the second exemplary embodiment, when the color conversion processis performed, the pixel sensitivity ratio of W:G:R:B is adjusted. Inaddition, like the second exemplary embodiment, the following two typesof color conversion process are provided: a color conversion processperformed at a time of full scanning and a color conversion processperformed at a time of pixel addition. In addition, the following twotypes of color conversion process are provided: a color conversionprocess 1 performed in a high luminance mode in which the luminance ofincident light is higher than a predetermined reference luminance and acolor conversion process 2 performed in a low luminance mode in whichthe luminance of incident light is lower than or equal to the referenceluminance.

Color Coding According to First Example of Third Exemplary Embodiment

A color conversion process performed in the case of color codingaccording to the first example of the third exemplary embodiment isdescribed next. First, the color conversion process 1 performed in ahigh luminance mode at a time of full scanning is described withreference to a flowchart shown in FIG. 54 and schematic illustrationsshown in FIGS. 55A to 55C.

FIG. 55A illustrates a 4×4 pixel color arrangement of the color codingaccording to the first example of the third exemplary embodiment. In thecolor coding according to the first example, as shown in FIG. 55B, thecomponents of the W pixels are expanded into pixels of all colors byusing the above-described existing signal process using directionalcorrelation and determining the direction of resolution (step S51).Thereafter, as shown in FIG. 55C, by using the above-described existingtechnique and a correlation between a W pixel and an R pixel, acorrelation between a W pixel and a G pixel, and a correlation between aW pixel and a B pixel, the R, G, and B components are expanded into allpixels (step S52).

Subsequently, the color conversion process 2 performed in a lowluminance mode at a time of full scanning is described with reference toa flowchart shown in FIG. 56 and schematic illustrations shown in FIGS.57A and 57B.

First, as shown in FIG. 57A, the direction of resolution is examined byusing signal processing based on the above-described directionalcorrelation (step S61). Thereafter, it is determined whether thedirection of resolution can be determined (step S62). If the directionof resolution can be determined, the components of the W pixels arrangedin a checkerboard pattern are expanded into pixels of all colors (stepS63).

Subsequently, as shown in FIG. 57B, R pixels and B pixels are generatedby using the above-described existing technique and the correlationbetween a W pixel and an R pixel and the correlation between a W pixeland a B pixel (step S64). However, if, in step S62, the direction ofresolution is not determined, each of the R and B pixels is generated byusing simple four-pixel averaging interpolation using four pixelsadjacent to the W pixel in the vertical direction and the horizontaldirection (step S67).

Subsequently, as shown in FIG. 57C, the signals of two G pixels adjacentto the W pixel are added to the signal of the W pixel (W+2G) whiledynamically changing the ratio. Thus, a checkerboard pattern is formed(step S65). Thereafter, the components are expanded into all pixels bydetermining the direction. In this way, a luminance signal is generated(step S66).

As described above, by using one of the color conversion process 1 andthe color conversion process 2 in accordance with the luminance of theincident light, the signals corresponding to a color arrangement inwhich W filters are arranged in a checkerboard pattern can be convertedinto signals corresponding to the RGB Bayer arrangement through signalprocessing performed by the DSP circuit 40 disposed outside the sensorchip 11.

The color conversion processes 1 and 2 performed at a time of pixeladdition for capturing a moving image are described next. First, thepixel addition process 1 is described with reference to a flowchartshown in FIG. 58 and schematic illustrations shown in FIGS. 59A to 59C.

As shown in FIG. 59A, addition is performed on two W pixels diagonallyadjacent to each other. In addition, for pixels of each of R, G, and Bcolors, counter addition is performed on two pixels arranged in adiagonal direction (step S71). Thereafter, as shown in FIG. 59B, Rcomponents, G components, and B components are expanded into all pixelsusing a correlation between a W pixel and an R pixel, a correlationbetween a W pixel and a G pixel, and a correlation between a W pixel anda B pixel (step S72). Subsequently, as shown in FIG. 59C, the signals ofthe W pixel and the G signal are added in proportion 1:2, and a (W+2G)signal is generated. The (W+2G) signals serve as luminance signals (stepS73).

The pixel addition process 2 is described next with reference to aflowchart shown in FIG. 60 and schematic illustrations shown in FIGS.61A to 61C.

First, as shown in FIG. 61A, for the W pixels and the G pixels, FDaddition is performed between two pixels arranged in the upperleft-bottom right diagonal directions and the upper right-bottom leftdiagonal direction, respectively. Thus, W and G pixels are generated(step S81). More specifically, for the G pixels, FD addition isperformed between a pixel of interest and a pixel located at the lowerleft of the pixel of interest (i.e., a pixel to the left by one columnand below by one row). For the W pixels, FD addition is performedbetween a pixel of interest and a pixel located at the lower right ofthe pixel of interest (i.e., a pixel to the right by one column andbelow by one row). The R and B pixels remain unchanged.

Subsequently, as shown in FIG. 61B, the R, G, and B components areexpanded into all pixels using the correlation between a W pixel and anR pixel, the correlation between a W pixel and a G pixel, and thecorrelation between a W pixel and a B pixel (step S82). Thereafter, asshown in FIG. 61C, the signals of the W pixel and the G signal are addedin proportion 1:2, and a (W+2G) signal is generated. The (W+2G) signalsserve as luminance signals (step S83).

As described above, by using one of the pixel addition process 1 and thepixel addition process 2 when a moving image is captured, the signalscorresponding to a color arrangement in which W filters are arranged ina checkerboard pattern can be converted into signals corresponding tothe RGB Bayer arrangement through signal processing performed by the DSPcircuit 40 disposed outside the sensor chip 11.

Color Coding According to Second Example of Third Exemplary Embodiment

A color conversion process for the color coding according to the secondexample of the third exemplary embodiment is described next. A series ofprocesses for the color coding according to the second example of thethird exemplary embodiment is generally based on that for the colorcoding according to the first example of the third exemplary embodiment.

The color conversion process 1 performed at a time of full scanning isdescribed with reference to schematic illustrations shown in FIGS. 62Aand 62B. FIG. 62A illustrates a color arrangement of color coding of a4×4 pixel block according to the second example of the third exemplaryembodiment. In the color coding according to the second example of thethird exemplary embodiment, as shown in FIG. 62B, the components of Wpixels arranged in a checkerboard pattern are expanded into pixels ofall colors using the direction of resolution. Subsequently, as shown inFIG. 62C, the R, G, and B components are expanded into all pixels usingthe correlation between a W pixel and an R pixel, the correlationbetween a W pixel and a G pixel, and the correlation between a W pixeland a B pixel.

The color conversion process 2 performed at a time of full scanning isdescribed next with reference to schematic illustrations shown in FIGS.63A to 63C. As in the color conversion process 1, in the colorarrangement of color coding of a 4×4 pixel block according to the secondexample, as shown in FIG. 63A, the components of W pixels arranged in acheckerboard pattern are expanded into pixels of all colors using thedirection of resolution.

Subsequently, as shown in FIG. 63B, the R and B components are expandedinto all pixels using the correlation between a W pixel and an R pixeland the correlation between a W pixel and a B pixel. Thereafter, asshown in FIG. 63C, by determining the direction using the W pixels, thesignals of two G pixels adjacent to the W pixel are added to the signalof the W pixel while dynamically changing the ratio (W+2G). Thus, acheckerboard pattern is formed. Thereafter, the components of the Gpixels are expanded into all pixels by further determining thedirection. In this way, a luminance signal is generated.

The pixel addition process 1 is described next with reference to FIGS.64A and 64B. As shown in FIG. 64A, for W pixels and G pixels, FDaddition is performed between two pixels arranged in the upperleft-bottom right diagonal directions and the upper right-bottom leftdiagonal directions, respectively. Thus, the signals of W, R, G, and Bpixels are generated. Thereafter, as shown in FIG. 64B, the R, G, and Bcomponents are expanded into all of the pixels. Subsequently, in orderto improve the S/N ratio, the components of the W pixels are fitted intoall of the pixels. Thus, R, G, and B signals are generated.

The pixel addition process 2 is described next with reference to FIGS.65A to 65C. As shown in FIG. 65A, for W pixels and G pixels, FD additionis performed between two pixels arranged in the upper left-bottom rightdiagonal directions and the upper right-bottom left diagonal directions,respectively. Thus, the signals of W, R, G, and B pixels are generated.Thereafter, as shown in FIG. 65B, the R, G, and B components areexpanded into all pixels using the correlation between a W pixel and anR pixel, the correlation between a W pixel and a G pixel, and thecorrelation between a W pixel and a B pixel. The signals of the W pixeland the G signal are then added in proportion 1:2, and a (W+2G) signalis generated. The (W+2G) signals serve as luminance signals.

As described above, according to the first and second examples, in thecolor coding having a color arrangement in which W filters of whiteserving as a primary color of a luminance signal are arranged in acheckerboard pattern and G filters are arranged so as to form a stripepattern in a diagonal direction, the following advantage can be providedby performing the above-described signal processing. That is, in the DSPcircuit 40 provided outside the sensor chip 11, by adding the signal ofa G pixel adjacent to a W pixel to the signal of the W pixel andperforming signal processing using the sum as a primary component of theluminance signal, the intensity of the luminance signal can beincreased. Accordingly, the sensitivity can be increased with a minimaldecrease in resolution.

3. Modifications

While the foregoing exemplary embodiments have been described withreference to a method in which the signals of two G pixels adjacent to aW pixel are simply added to the signal of the W pixel and the sum isused as a primary component of a luminance signal, the number of the Gpixels used for addition is not limited to two. The number of the Gpixels may be one. Note that 2-pixel addition in which the signal of oneG pixel adjacent to a W pixel is added to the signal of the W pixel hasan advantage over 3-pixel addition in which the signals of two G pixelsadjacent to a W pixel are added to the signal of the W pixel in that itcan reduce a decrease in resolution.

In addition, while the foregoing exemplary embodiments have beendescribed with reference to signal processing in the case of a diagonalstripe pattern of a series of four pixels of G filters, the sameadvantage can be provided even in the case in which a series of fourpixels of G filters is repeatedly arranged in the vertical direction orthe horizontal direction so as to form a stripe pattern. Furthermore,the addition ratio of the signal of a filter of a first color to that ofa second color is not limited to 1:1. Any addition ratio that balancesthe resolution with the sensitivity can be used.

Still furthermore, while the foregoing exemplary embodiments have beendescribed with reference to the color coding in which the first colorserving as a primary color of a luminance signal is white (W) and thesecond color is green (G), the same advantage can be provided even forthe color coding in which the first color is green and the second coloris white. Such color coding according to the first example and secondexample is shown in FIGS. 66 and 67, respectively.

Example of Application Imaging Apparatus

FIG. 68 is a block diagram of an exemplary configuration of an imagingapparatus according to an embodiment of the present invention.

As shown in FIG. 68, according to the embodiment of the presentinvention, an imaging apparatus 100 includes an optical system includinga lens unit 101, an image pickup device 102, a DSP circuit 103 servingas a camera signal processing circuit, a frame memory 104, a displayunit 105, a recording unit 106, an operation unit 107, and a powersupply unit 108. The DSP circuit 103, the frame memory 104, the displayunit 105, the recording unit 106, the operation unit 107, and the powersupply unit 108 are connected to one another via a bus line 109.

The lens unit 101 receives incident light (imaging light) output from asubject and forms an image on an imaging surface of the image pickupdevice 102. The image pickup device 102 converts the intensity of theincident light that is received by the lens unit 101 and that forms animage on the imaging surface into an electrical signal for each pixel.The electrical signals are output in the form of pixel signals. The CMOSimage sensors 10A and 10B according to the first and second exemplaryembodiments can be used as the image pickup device 102. In addition, theCMOS image sensor 10 according to all of the above-described exemplaryembodiments (including the third exemplary embodiment) can be used asthe image pickup device 102.

In the CMOS image sensors 10A, 10B, and 10, the color filter arrayemploys color coding having a color arrangement in which a color servingas a primary color of a luminance signal is arranged in a checkerboardpattern, and a plurality of colors serving as color informationcomponents are arranged in the other area of the checkerboard pattern.In particular, the CMOS image sensors 10A and 10 convert signalscorresponding to the color arrangement of the color filter array intosignals corresponding to the Bayer arrangement through computationperformed in the sensor chip 11.

Accordingly, although the CMOS image sensors 10A and 10 have the colorfilter array using color coding in which a color serving as a primarycolor of a luminance signal is arranged in a checkerboard pattern, theCMOS image sensors 10A and 10 output signals corresponding to the Bayerarrangement. Consequently, an existing DSP for the Bayer arrangementthat generates a luminance signal Y and two color difference signals U(B−Y) and V (R−Y) using signals corresponding to the Bayer arrangementcan be used as the DSP circuit 103.

Since, as described above, an existing DSP for the Bayer arrangement canbe used, the development of a new DSP that is significantly costly isnot necessary even when the color coding of a color filter array used inthe image pickup device 102 is changed. The use of an existing DSP cancontribute to cost reduction in the production of the imaging apparatus100 including the DSP circuit 103 and, in particular, to widespread useof a color filter array having color coding using a W filter.

In contrast, in the case of the CMOS image sensor 10B, signalscorresponding to a color arrangement in which a color serving as aprimary color of a luminance signal is arranged in a checkerboardpattern are output to outside the chip, and the DSP circuit 103(corresponding to the DSP circuit 40 shown in FIG. 52) converts thesignals into signals corresponding to the Bayer arrangement.Accordingly, when the color coding of the color filter array is changed,the development of a new DSP circuit 103 is necessary and, therefore,the development cost is necessary. However, like the CMOS image sensor10A, the CMOS image sensor 10B can increase the sensitivity with aminimal decrease in resolution. Accordingly, the CMOS image sensor 10Bhas an advantage in that it can increase the S/N ratio of an imagingsignal while maintaining a high resolution.

The display unit 105 is a panel display unit, such as a liquid crystaldisplay or an organic electroluminescence (EL) display. The display unit105 displays a moving image or a still image captured by the imagepickup device 102. The recording unit 106 records a moving image or astill image captured by the image pickup device 102 on a recordingmedium, such as a video tape or a digital versatile disk (DVD).

Under the control of a user, the operation unit 107 submits a variety ofoperation commands that control a variety of functions of the imagingapparatus. The power supply unit 108 supplies electrical power to theDSP circuit 103, the frame memory 104, the display unit 105, therecording unit 106, and the operation unit 107 when necessary.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An electronic imaging apparatus, comprising: alens system operative to gather light; and a solid-state imaging devicepositioned to receive light from the lens system, the solid-stateimaging device including: a pixel array unit, including: a lightreceiving surface; a plurality of unit pixels; a color filter array unitdisposed on a light incident side of the pixel array unit, the colorfilter array including a plurality of white light filters, a pluralityof green light filters, a plurality of red light filters, and aplurality of blue light filters, wherein the plurality of white lightfilters are arranged in a checkerboard pattern, and wherein along adiagonal line of color filters in the color filter array unit, the colorfilters include a group of two blue color filters adjacent to a group oftwo red color filters; and a conversion processing unit configured toreceive output signals from the pixel array unit.
 2. The electronicimaging apparatus of claim 1, wherein along a second diagonal line ofcolor filters in the color filter array unit, the color filters includeat least two green color filters, and wherein along a third diagonalline of color filters in the color filter array unit, the color filtersinclude at least two green color filters.
 3. The electronic imagingapparatus of claim 2, wherein in a 4×4 group of color filters, a firstcorner is a white color filter, a second corner is a red color filter, athird corner is a blue color filter, and a fourth corner is a whitecolor filter.
 4. The electronic imaging apparatus of claim 1, whereinfour unit pixels share at least one of a floating diffusion unit, aselection transistor, and an amplifying transistor.
 5. The electronicimaging apparatus of claim 1, wherein the color conversion processingunit is configured receive the output signals from the pixel array unitcorresponding to a color filter arrangement of the color filter array,convert the received signals into signals corresponding to a Bayerarrangement, and output the signals corresponding to the Bayerarrangement.
 6. The electronic apparatus of claim 5, wherein a firstcolor conversion process is performed when a luminance of incident lightis greater than a predetermined threshold, and a second color conversionprocess is performed when a luminance of incident light is less than apredetermined threshold.
 7. The electronic apparatus of claim 6, whereinin the first color conversion process, signals corresponding to whitelight filters are replaced with green color pixels; red pixels aregenerated according to a correlation between signals corresponding towhite light filters and signals corresponding to red light filters; andblue pixels are generated according to a correlation between signalscorresponding to white light filters and signals corresponding to bluelight filters.
 8. The electronic apparatus of claim 6, wherein in thesecond color conversion process, red pixels are generated according to acorrelation between signals corresponding to white light filters andsignals corresponding to red light filters; blue pixels are generatedaccording to a correlation between signals corresponding to white lightfilters and signals corresponding to blue light filters; and greenpixels are generated by adding a signal corresponding to a white lightfilter and signals corresponding to green light filters.
 9. Theelectronic apparatus of claim 6, wherein in the second color conversionprocess, each of the red pixels and green pixels is generated for theBayer arrangement utilizing four-pixel averaging interpolation usingfour pixels adjacent to the white pixel in the vertical direction andthe horizontal.
 10. The electronic apparatus of claim 5, wherein firstand second signals from first and second unit pixels corresponding towhite light filters are added together at a floating diffusion regionshared by the first and second unit pixels, and wherein the first andsecond unit pixels are diagonally adjacent to each other.
 11. Theelectronic apparatus of claim 10, wherein for each red, green, and bluepixel of the Bayer arrangement, performing pixel addition on two pixelsarranged with one pixel therebetween in an opposite diagonal direction.12. A signal processing method, comprising: capturing light with a pixelarray unit having a color filter unit through which the light isfiltered, the pixel array unit having pixels arranged two-dimensionallyin a matrix of rows and columns relatively along a first direction and asecond direction perpendicular to the first direction respectively, thecolor filter unit disposed over the pixel array unit, the color filterunit having a coding with regions respectively corresponding to thepixels, the regions transmitting at least one of white light, greenlight, red light, and blue light, wherein the plurality of white lightfilters are arranged in a checkerboard pattern, and wherein along adiagonal line of color filters in the color filter array unit, the colorfilters include a group of two blue color filters adjacent to a group oftwo red color filters; receiving the output signals from the pixel arrayunit corresponding to a color filter arrangement of the color filterarray; converting the received signals into signals corresponding to aBayer arrangement; and outputting the signals corresponding to the Bayerarrangement.
 13. The signal processing method of claim 12, wherein alonga second diagonal line of color filters in the color filter array unit,the color filters include at least two green color filters, and whereinalong a third diagonal line of color filters in the color filter arrayunit, the color filters include at least two green color filters. 14.The signal processing method of claim 13, wherein in a 4×4 group ofcolor filters, a first corner is a white color filter, a second corneris a red color filter, a third corner is a blue color filter, and afourth corner is a white.
 15. The signal processing method of claim 12,further comprising: performing a first color conversion process when aluminance of incident light is greater than a predetermined threshold;and performing a second color conversion process when the luminance ofincident light is less than a predetermined threshold.
 16. The signalprocessing method of claim 15, wherein the first color conversionprocess includes: replacing signals corresponding to white light filterswith green color pixels; generating red pixels according to acorrelation between signals corresponding to white light filters andsignals corresponding to red light filters; and generating blue pixelsaccording to a correlation between signals corresponding to white lightfilters and signals corresponding to blue light filters.
 17. The signalprocessing method of claim 15, wherein the second color conversionprocess includes: generating red pixels according to a correlationbetween signals corresponding to white light filters and signalscorresponding to red light filters; generating blue pixels according toa correlation between signals corresponding to white light filters andsignals corresponding to blue light filters; and generating green pixelsby adding a signal corresponding to a white light filter and signalscorresponding to green light filters.
 18. The signal processing methodof claim 15, wherein the second color conversion process includes:generating each of the red pixels and green pixels of the Bayerarrangement utilizing four-pixel averaging interpolation using fourpixels adjacent to the white pixel in the vertical direction and thehorizontal.
 19. The signal processing method of claim 12, furthercomprising: adding together first and second signals from first andsecond unit pixels corresponding to white light filters at a floatingdiffusion region shared by the first and second unit pixels, wherein thefirst and second unit pixels are diagonally adjacent to each other. 20.The signal processing method of claim 19, further comprising: for eachred, green, and blue pixel of the Bayer arrangement, performing pixeladdition on two pixels arranged with one pixel therebetween in anopposite diagonal direction.